2023-02-09 14:53:57,592 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-02-09 14:53:57,825 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe 2023-02-09 14:53:57,839 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-09 14:53:57,840 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/cluster_fa49ce5e-dcbf-0fd7-6b70-a45a30f7cf80, deleteOnExit=true 2023-02-09 14:53:57,840 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-09 14:53:57,840 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/test.cache.data in system properties and HBase conf 2023-02-09 14:53:57,841 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/hadoop.tmp.dir in system properties and HBase conf 2023-02-09 14:53:57,841 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/hadoop.log.dir in system properties and HBase conf 2023-02-09 14:53:57,842 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-09 14:53:57,842 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-09 14:53:57,842 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-09 14:53:57,959 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-09 14:53:58,335 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-09 14:53:58,339 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-09 14:53:58,340 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-09 14:53:58,340 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-09 14:53:58,340 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-09 14:53:58,340 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-09 14:53:58,341 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-09 14:53:58,341 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-09 14:53:58,341 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-09 14:53:58,342 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-09 14:53:58,342 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/nfs.dump.dir in system properties and HBase conf 2023-02-09 14:53:58,343 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/java.io.tmpdir in system properties and HBase conf 2023-02-09 14:53:58,343 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-09 14:53:58,343 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-09 14:53:58,343 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-09 14:53:58,845 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-09 14:53:58,849 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-09 14:53:59,138 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-09 14:53:59,292 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-09 14:53:59,306 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 14:53:59,342 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 14:53:59,372 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/java.io.tmpdir/Jetty_localhost_32867_hdfs____dddm1w/webapp 2023-02-09 14:53:59,527 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:32867 2023-02-09 14:53:59,537 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-09 14:53:59,537 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-09 14:53:59,909 WARN [Listener at localhost/38049] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 14:54:00,006 WARN [Listener at localhost/38049] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-09 14:54:00,024 WARN [Listener at localhost/38049] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 14:54:00,030 INFO [Listener at localhost/38049] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 14:54:00,035 INFO [Listener at localhost/38049] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/java.io.tmpdir/Jetty_localhost_33555_datanode____o0t09o/webapp 2023-02-09 14:54:00,129 INFO [Listener at localhost/38049] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33555 2023-02-09 14:54:00,446 WARN [Listener at localhost/41631] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 14:54:00,458 WARN [Listener at localhost/41631] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-09 14:54:00,462 WARN [Listener at localhost/41631] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 14:54:00,464 INFO [Listener at localhost/41631] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 14:54:00,470 INFO [Listener at localhost/41631] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/java.io.tmpdir/Jetty_localhost_39329_datanode____7av0b/webapp 2023-02-09 14:54:00,575 INFO [Listener at localhost/41631] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39329 2023-02-09 14:54:00,583 WARN [Listener at localhost/33247] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 14:54:00,595 WARN [Listener at localhost/33247] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-09 14:54:00,598 WARN [Listener at localhost/33247] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-09 14:54:00,600 INFO [Listener at localhost/33247] log.Slf4jLog(67): jetty-6.1.26 2023-02-09 14:54:00,607 INFO [Listener at localhost/33247] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/java.io.tmpdir/Jetty_localhost_34933_datanode____.bsb7l/webapp 2023-02-09 14:54:00,714 INFO [Listener at localhost/33247] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34933 2023-02-09 14:54:00,724 WARN [Listener at localhost/40823] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-09 14:54:00,921 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc25fbbc1408d13d2: Processing first storage report for DS-54d025cc-513f-4ba6-87d1-26db3b9f651c from datanode 30e687f1-c071-43d7-89db-78c265b7458a 2023-02-09 14:54:00,922 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc25fbbc1408d13d2: from storage DS-54d025cc-513f-4ba6-87d1-26db3b9f651c node DatanodeRegistration(127.0.0.1:45449, datanodeUuid=30e687f1-c071-43d7-89db-78c265b7458a, infoPort=41347, infoSecurePort=0, ipcPort=40823, storageInfo=lv=-57;cid=testClusterID;nsid=2126837550;c=1675954438937), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-09 14:54:00,923 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd37a6502197879c8: Processing first storage report for DS-0c314c2a-baf0-4782-8322-f37f2df23641 from datanode f8e8dad4-c7b5-422d-a273-bab2cf112e44 2023-02-09 14:54:00,923 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd37a6502197879c8: from storage DS-0c314c2a-baf0-4782-8322-f37f2df23641 node DatanodeRegistration(127.0.0.1:38383, datanodeUuid=f8e8dad4-c7b5-422d-a273-bab2cf112e44, infoPort=34329, infoSecurePort=0, ipcPort=33247, storageInfo=lv=-57;cid=testClusterID;nsid=2126837550;c=1675954438937), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 14:54:00,923 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x9cf6884a71d9b6c8: Processing first storage report for DS-4e02748c-1ef7-4f5d-96f6-080237b6b997 from datanode 89217ea8-d9db-4980-9a81-2c27c9377629 2023-02-09 14:54:00,923 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x9cf6884a71d9b6c8: from storage DS-4e02748c-1ef7-4f5d-96f6-080237b6b997 node DatanodeRegistration(127.0.0.1:35385, datanodeUuid=89217ea8-d9db-4980-9a81-2c27c9377629, infoPort=45909, infoSecurePort=0, ipcPort=41631, storageInfo=lv=-57;cid=testClusterID;nsid=2126837550;c=1675954438937), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 14:54:00,923 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc25fbbc1408d13d2: Processing first storage report for DS-ce537963-9ec8-4f84-90dc-5a3b47cc23e1 from datanode 30e687f1-c071-43d7-89db-78c265b7458a 2023-02-09 14:54:00,923 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc25fbbc1408d13d2: from storage DS-ce537963-9ec8-4f84-90dc-5a3b47cc23e1 node DatanodeRegistration(127.0.0.1:45449, datanodeUuid=30e687f1-c071-43d7-89db-78c265b7458a, infoPort=41347, infoSecurePort=0, ipcPort=40823, storageInfo=lv=-57;cid=testClusterID;nsid=2126837550;c=1675954438937), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 14:54:00,923 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xd37a6502197879c8: Processing first storage report for DS-6002ae10-56a6-4b22-9223-9041df2b6218 from datanode f8e8dad4-c7b5-422d-a273-bab2cf112e44 2023-02-09 14:54:00,923 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xd37a6502197879c8: from storage DS-6002ae10-56a6-4b22-9223-9041df2b6218 node DatanodeRegistration(127.0.0.1:38383, datanodeUuid=f8e8dad4-c7b5-422d-a273-bab2cf112e44, infoPort=34329, infoSecurePort=0, ipcPort=33247, storageInfo=lv=-57;cid=testClusterID;nsid=2126837550;c=1675954438937), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 14:54:00,924 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x9cf6884a71d9b6c8: Processing first storage report for DS-a36d6a90-90ec-4046-bf83-b28c0c9416d4 from datanode 89217ea8-d9db-4980-9a81-2c27c9377629 2023-02-09 14:54:00,924 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x9cf6884a71d9b6c8: from storage DS-a36d6a90-90ec-4046-bf83-b28c0c9416d4 node DatanodeRegistration(127.0.0.1:35385, datanodeUuid=89217ea8-d9db-4980-9a81-2c27c9377629, infoPort=45909, infoSecurePort=0, ipcPort=41631, storageInfo=lv=-57;cid=testClusterID;nsid=2126837550;c=1675954438937), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-09 14:54:01,115 DEBUG [Listener at localhost/40823] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe 2023-02-09 14:54:01,174 INFO [Listener at localhost/40823] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/cluster_fa49ce5e-dcbf-0fd7-6b70-a45a30f7cf80/zookeeper_0, clientPort=52509, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/cluster_fa49ce5e-dcbf-0fd7-6b70-a45a30f7cf80/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/cluster_fa49ce5e-dcbf-0fd7-6b70-a45a30f7cf80/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-09 14:54:01,190 INFO [Listener at localhost/40823] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=52509 2023-02-09 14:54:01,199 INFO [Listener at localhost/40823] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 14:54:01,202 INFO [Listener at localhost/40823] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 14:54:01,860 INFO [Listener at localhost/40823] util.FSUtils(479): Created version file at hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9 with version=8 2023-02-09 14:54:01,861 INFO [Listener at localhost/40823] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/hbase-staging 2023-02-09 14:54:02,149 INFO [Listener at localhost/40823] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-09 14:54:02,586 INFO [Listener at localhost/40823] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-02-09 14:54:02,616 INFO [Listener at localhost/40823] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 14:54:02,617 INFO [Listener at localhost/40823] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 14:54:02,617 INFO [Listener at localhost/40823] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 14:54:02,618 INFO [Listener at localhost/40823] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 14:54:02,618 INFO [Listener at localhost/40823] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 14:54:02,754 INFO [Listener at localhost/40823] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 14:54:02,832 DEBUG [Listener at localhost/40823] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-09 14:54:02,926 INFO [Listener at localhost/40823] ipc.NettyRpcServer(120): Bind to /172.31.14.131:43829 2023-02-09 14:54:02,937 INFO [Listener at localhost/40823] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 14:54:02,939 INFO [Listener at localhost/40823] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 14:54:02,959 INFO [Listener at localhost/40823] zookeeper.RecoverableZooKeeper(93): Process identifier=master:43829 connecting to ZooKeeper ensemble=127.0.0.1:52509 2023-02-09 14:54:03,000 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:438290x0, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 14:54:03,006 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:43829-0x1003993358c0000 connected 2023-02-09 14:54:03,042 DEBUG [Listener at localhost/40823] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 14:54:03,043 DEBUG [Listener at localhost/40823] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 14:54:03,047 DEBUG [Listener at localhost/40823] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 14:54:03,055 DEBUG [Listener at localhost/40823] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43829 2023-02-09 14:54:03,056 DEBUG [Listener at localhost/40823] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43829 2023-02-09 14:54:03,056 DEBUG [Listener at localhost/40823] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43829 2023-02-09 14:54:03,059 DEBUG [Listener at localhost/40823] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43829 2023-02-09 14:54:03,060 DEBUG [Listener at localhost/40823] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43829 2023-02-09 14:54:03,066 INFO [Listener at localhost/40823] master.HMaster(439): hbase.rootdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9, hbase.cluster.distributed=false 2023-02-09 14:54:03,134 INFO [Listener at localhost/40823] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-09 14:54:03,134 INFO [Listener at localhost/40823] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 14:54:03,134 INFO [Listener at localhost/40823] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 14:54:03,135 INFO [Listener at localhost/40823] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 14:54:03,135 INFO [Listener at localhost/40823] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 14:54:03,135 INFO [Listener at localhost/40823] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 14:54:03,139 INFO [Listener at localhost/40823] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 14:54:03,143 INFO [Listener at localhost/40823] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33423 2023-02-09 14:54:03,145 INFO [Listener at localhost/40823] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-09 14:54:03,151 DEBUG [Listener at localhost/40823] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-09 14:54:03,152 INFO [Listener at localhost/40823] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 14:54:03,154 INFO [Listener at localhost/40823] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 14:54:03,156 INFO [Listener at localhost/40823] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33423 connecting to ZooKeeper ensemble=127.0.0.1:52509 2023-02-09 14:54:03,162 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:334230x0, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 14:54:03,162 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:33423-0x1003993358c0001 connected 2023-02-09 14:54:03,163 DEBUG [Listener at localhost/40823] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 14:54:03,164 DEBUG [Listener at localhost/40823] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 14:54:03,165 DEBUG [Listener at localhost/40823] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 14:54:03,166 DEBUG [Listener at localhost/40823] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33423 2023-02-09 14:54:03,166 DEBUG [Listener at localhost/40823] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33423 2023-02-09 14:54:03,168 DEBUG [Listener at localhost/40823] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33423 2023-02-09 14:54:03,168 DEBUG [Listener at localhost/40823] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33423 2023-02-09 14:54:03,168 DEBUG [Listener at localhost/40823] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33423 2023-02-09 14:54:03,181 INFO [Listener at localhost/40823] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-09 14:54:03,181 INFO [Listener at localhost/40823] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 14:54:03,181 INFO [Listener at localhost/40823] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 14:54:03,182 INFO [Listener at localhost/40823] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 14:54:03,182 INFO [Listener at localhost/40823] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 14:54:03,182 INFO [Listener at localhost/40823] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 14:54:03,182 INFO [Listener at localhost/40823] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 14:54:03,184 INFO [Listener at localhost/40823] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33553 2023-02-09 14:54:03,184 INFO [Listener at localhost/40823] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-09 14:54:03,185 DEBUG [Listener at localhost/40823] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-09 14:54:03,186 INFO [Listener at localhost/40823] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 14:54:03,188 INFO [Listener at localhost/40823] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 14:54:03,190 INFO [Listener at localhost/40823] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33553 connecting to ZooKeeper ensemble=127.0.0.1:52509 2023-02-09 14:54:03,194 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:335530x0, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 14:54:03,195 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:33553-0x1003993358c0002 connected 2023-02-09 14:54:03,195 DEBUG [Listener at localhost/40823] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 14:54:03,196 DEBUG [Listener at localhost/40823] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 14:54:03,196 DEBUG [Listener at localhost/40823] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 14:54:03,197 DEBUG [Listener at localhost/40823] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33553 2023-02-09 14:54:03,198 DEBUG [Listener at localhost/40823] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33553 2023-02-09 14:54:03,198 DEBUG [Listener at localhost/40823] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33553 2023-02-09 14:54:03,199 DEBUG [Listener at localhost/40823] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33553 2023-02-09 14:54:03,199 DEBUG [Listener at localhost/40823] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33553 2023-02-09 14:54:03,212 INFO [Listener at localhost/40823] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-02-09 14:54:03,213 INFO [Listener at localhost/40823] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 14:54:03,213 INFO [Listener at localhost/40823] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-09 14:54:03,213 INFO [Listener at localhost/40823] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-09 14:54:03,213 INFO [Listener at localhost/40823] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-09 14:54:03,213 INFO [Listener at localhost/40823] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-09 14:54:03,213 INFO [Listener at localhost/40823] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-09 14:54:03,215 INFO [Listener at localhost/40823] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41077 2023-02-09 14:54:03,215 INFO [Listener at localhost/40823] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-09 14:54:03,216 DEBUG [Listener at localhost/40823] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-09 14:54:03,218 INFO [Listener at localhost/40823] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 14:54:03,219 INFO [Listener at localhost/40823] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 14:54:03,220 INFO [Listener at localhost/40823] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41077 connecting to ZooKeeper ensemble=127.0.0.1:52509 2023-02-09 14:54:03,225 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:410770x0, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-09 14:54:03,226 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:41077-0x1003993358c0003 connected 2023-02-09 14:54:03,226 DEBUG [Listener at localhost/40823] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 14:54:03,227 DEBUG [Listener at localhost/40823] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 14:54:03,227 DEBUG [Listener at localhost/40823] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-09 14:54:03,228 DEBUG [Listener at localhost/40823] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41077 2023-02-09 14:54:03,228 DEBUG [Listener at localhost/40823] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41077 2023-02-09 14:54:03,228 DEBUG [Listener at localhost/40823] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41077 2023-02-09 14:54:03,229 DEBUG [Listener at localhost/40823] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41077 2023-02-09 14:54:03,229 DEBUG [Listener at localhost/40823] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41077 2023-02-09 14:54:03,231 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,43829,1675954441993 2023-02-09 14:54:03,241 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-09 14:54:03,242 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,43829,1675954441993 2023-02-09 14:54:03,261 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 14:54:03,261 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 14:54:03,261 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 14:54:03,261 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-09 14:54:03,263 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 14:54:03,263 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-09 14:54:03,264 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,43829,1675954441993 from backup master directory 2023-02-09 14:54:03,265 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-09 14:54:03,269 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,43829,1675954441993 2023-02-09 14:54:03,269 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-09 14:54:03,269 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 14:54:03,269 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,43829,1675954441993 2023-02-09 14:54:03,272 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-09 14:54:03,273 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-09 14:54:03,370 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/hbase.id with ID: 37eb581a-c31d-4e36-ac9c-3ff37aae9eef 2023-02-09 14:54:03,417 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-09 14:54:03,436 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 14:54:03,492 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x555f3efe to 127.0.0.1:52509 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:54:03,526 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@32a90e6a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:54:03,549 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-09 14:54:03,551 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-09 14:54:03,568 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-09 14:54:03,568 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-09 14:54:03,570 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-09 14:54:03,574 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-09 14:54:03,575 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 14:54:03,608 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/MasterData/data/master/store-tmp 2023-02-09 14:54:03,643 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:03,644 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-09 14:54:03,644 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 14:54:03,644 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 14:54:03,644 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-09 14:54:03,644 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 14:54:03,644 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 14:54:03,645 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675954443644Disabling compacts and flushes for region at 1675954443644Disabling writes for close at 1675954443644Writing region close event to WAL at 1675954443644Closed at 1675954443644 2023-02-09 14:54:03,646 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/MasterData/WALs/jenkins-hbase4.apache.org,43829,1675954441993 2023-02-09 14:54:03,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C43829%2C1675954441993, suffix=, logDir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/MasterData/WALs/jenkins-hbase4.apache.org,43829,1675954441993, archiveDir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/MasterData/oldWALs, maxLogs=10 2023-02-09 14:54:03,721 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45449,DS-54d025cc-513f-4ba6-87d1-26db3b9f651c,DISK] 2023-02-09 14:54:03,721 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38383,DS-0c314c2a-baf0-4782-8322-f37f2df23641,DISK] 2023-02-09 14:54:03,721 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35385,DS-4e02748c-1ef7-4f5d-96f6-080237b6b997,DISK] 2023-02-09 14:54:03,729 DEBUG [RS-EventLoopGroup-5-1] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-09 14:54:03,803 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/MasterData/WALs/jenkins-hbase4.apache.org,43829,1675954441993/jenkins-hbase4.apache.org%2C43829%2C1675954441993.1675954443676 2023-02-09 14:54:03,804 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38383,DS-0c314c2a-baf0-4782-8322-f37f2df23641,DISK], DatanodeInfoWithStorage[127.0.0.1:45449,DS-54d025cc-513f-4ba6-87d1-26db3b9f651c,DISK], DatanodeInfoWithStorage[127.0.0.1:35385,DS-4e02748c-1ef7-4f5d-96f6-080237b6b997,DISK]] 2023-02-09 14:54:03,805 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:54:03,805 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:03,809 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 14:54:03,811 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 14:54:03,890 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-09 14:54:03,898 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-09 14:54:03,923 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-09 14:54:03,935 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:03,941 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-09 14:54:03,943 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-09 14:54:03,956 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-09 14:54:03,960 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:54:03,961 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60811880, jitterRate=-0.09383237361907959}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 14:54:03,961 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1675954443848Initializing all the Stores at 1675954443850 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954443851 (+1 ms)Cleaning up temporary data from old regions at 1675954443948 (+97 ms)Cleaning up detritus from prior splits at 1675954443949 (+1 ms)Region opened successfully at 1675954443961 (+12 ms) 2023-02-09 14:54:03,962 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-09 14:54:03,982 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:03,982 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:03,984 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:03,986 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-09 14:54:04,018 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 32 msec 2023-02-09 14:54:04,018 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-09 14:54:04,043 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-09 14:54:04,048 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-09 14:54:04,072 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-09 14:54:04,076 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-09 14:54:04,077 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-09 14:54:04,082 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-09 14:54:04,086 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-09 14:54:04,089 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 14:54:04,090 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-09 14:54:04,091 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-09 14:54:04,106 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-09 14:54:04,111 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 14:54:04,111 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 14:54:04,111 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 14:54:04,111 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-09 14:54:04,111 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 14:54:04,114 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,43829,1675954441993, sessionid=0x1003993358c0000, setting cluster-up flag (Was=false) 2023-02-09 14:54:04,128 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 14:54:04,134 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-09 14:54:04,135 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,43829,1675954441993 2023-02-09 14:54:04,140 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 14:54:04,146 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-09 14:54:04,147 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,43829,1675954441993 2023-02-09 14:54:04,150 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.hbase-snapshot/.tmp 2023-02-09 14:54:04,233 INFO [RS:0;jenkins-hbase4:33423] regionserver.HRegionServer(952): ClusterId : 37eb581a-c31d-4e36-ac9c-3ff37aae9eef 2023-02-09 14:54:04,233 INFO [RS:1;jenkins-hbase4:33553] regionserver.HRegionServer(952): ClusterId : 37eb581a-c31d-4e36-ac9c-3ff37aae9eef 2023-02-09 14:54:04,233 INFO [RS:2;jenkins-hbase4:41077] regionserver.HRegionServer(952): ClusterId : 37eb581a-c31d-4e36-ac9c-3ff37aae9eef 2023-02-09 14:54:04,240 DEBUG [RS:1;jenkins-hbase4:33553] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-09 14:54:04,240 DEBUG [RS:2;jenkins-hbase4:41077] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-09 14:54:04,240 DEBUG [RS:0;jenkins-hbase4:33423] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-09 14:54:04,248 DEBUG [RS:2;jenkins-hbase4:41077] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-09 14:54:04,248 DEBUG [RS:1;jenkins-hbase4:33553] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-09 14:54:04,248 DEBUG [RS:0;jenkins-hbase4:33423] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-09 14:54:04,248 DEBUG [RS:1;jenkins-hbase4:33553] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-09 14:54:04,248 DEBUG [RS:2;jenkins-hbase4:41077] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-09 14:54:04,248 DEBUG [RS:0;jenkins-hbase4:33423] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-09 14:54:04,256 DEBUG [RS:0;jenkins-hbase4:33423] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-09 14:54:04,256 DEBUG [RS:1;jenkins-hbase4:33553] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-09 14:54:04,256 DEBUG [RS:2;jenkins-hbase4:41077] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-09 14:54:04,258 DEBUG [RS:0;jenkins-hbase4:33423] zookeeper.ReadOnlyZKClient(139): Connect 0x25af0fff to 127.0.0.1:52509 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:54:04,259 DEBUG [RS:1;jenkins-hbase4:33553] zookeeper.ReadOnlyZKClient(139): Connect 0x49fba862 to 127.0.0.1:52509 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:54:04,259 DEBUG [RS:2;jenkins-hbase4:41077] zookeeper.ReadOnlyZKClient(139): Connect 0x749237b6 to 127.0.0.1:52509 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:54:04,286 DEBUG [RS:0;jenkins-hbase4:33423] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2b68e53a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:54:04,287 DEBUG [RS:2;jenkins-hbase4:41077] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7a55d754, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:54:04,287 DEBUG [RS:1;jenkins-hbase4:33553] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@473c4f2d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:54:04,287 DEBUG [RS:2;jenkins-hbase4:41077] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7e6a5cc6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-09 14:54:04,287 DEBUG [RS:0;jenkins-hbase4:33423] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2f57648f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-09 14:54:04,287 DEBUG [RS:1;jenkins-hbase4:33553] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4303c312, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-09 14:54:04,291 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:04,302 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 14:54:04,302 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 14:54:04,302 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 14:54:04,302 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-09 14:54:04,302 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-09 14:54:04,303 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,303 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-09 14:54:04,303 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,304 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954474304 2023-02-09 14:54:04,307 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-09 14:54:04,315 DEBUG [RS:1;jenkins-hbase4:33553] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:33553 2023-02-09 14:54:04,317 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:04,317 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-09 14:54:04,315 DEBUG [RS:2;jenkins-hbase4:41077] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:41077 2023-02-09 14:54:04,321 INFO [RS:1;jenkins-hbase4:33553] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-09 14:54:04,321 INFO [RS:1;jenkins-hbase4:33553] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-09 14:54:04,322 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-09 14:54:04,322 DEBUG [RS:1;jenkins-hbase4:33553] regionserver.HRegionServer(1023): About to register with Master. 2023-02-09 14:54:04,322 INFO [RS:2;jenkins-hbase4:41077] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-09 14:54:04,323 INFO [RS:2;jenkins-hbase4:41077] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-09 14:54:04,323 DEBUG [RS:2;jenkins-hbase4:41077] regionserver.HRegionServer(1023): About to register with Master. 2023-02-09 14:54:04,325 DEBUG [RS:0;jenkins-hbase4:33423] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:33423 2023-02-09 14:54:04,325 INFO [RS:0;jenkins-hbase4:33423] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-09 14:54:04,325 INFO [RS:0;jenkins-hbase4:33423] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-09 14:54:04,325 DEBUG [RS:0;jenkins-hbase4:33423] regionserver.HRegionServer(1023): About to register with Master. 2023-02-09 14:54:04,326 INFO [RS:1;jenkins-hbase4:33553] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,43829,1675954441993 with isa=jenkins-hbase4.apache.org/172.31.14.131:33553, startcode=1675954443180 2023-02-09 14:54:04,326 INFO [RS:0;jenkins-hbase4:33423] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,43829,1675954441993 with isa=jenkins-hbase4.apache.org/172.31.14.131:33423, startcode=1675954443133 2023-02-09 14:54:04,328 INFO [RS:2;jenkins-hbase4:41077] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,43829,1675954441993 with isa=jenkins-hbase4.apache.org/172.31.14.131:41077, startcode=1675954443212 2023-02-09 14:54:04,329 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-09 14:54:04,331 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-09 14:54:04,331 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-09 14:54:04,332 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-09 14:54:04,332 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-09 14:54:04,333 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:04,336 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-09 14:54:04,338 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-09 14:54:04,338 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-09 14:54:04,341 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-09 14:54:04,342 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-09 14:54:04,344 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675954444344,5,FailOnTimeoutGroup] 2023-02-09 14:54:04,344 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675954444344,5,FailOnTimeoutGroup] 2023-02-09 14:54:04,344 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:04,344 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-09 14:54:04,345 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:04,346 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:04,347 DEBUG [RS:1;jenkins-hbase4:33553] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-09 14:54:04,347 DEBUG [RS:2;jenkins-hbase4:41077] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-09 14:54:04,347 DEBUG [RS:0;jenkins-hbase4:33423] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-09 14:54:04,440 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-09 14:54:04,441 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-09 14:54:04,442 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9 2023-02-09 14:54:04,445 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33165, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-09 14:54:04,451 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45305, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-09 14:54:04,452 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37829, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-09 14:54:04,485 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:04,485 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43829] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:04,490 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43829] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:04,523 DEBUG [RS:2;jenkins-hbase4:41077] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9 2023-02-09 14:54:04,523 DEBUG [RS:2;jenkins-hbase4:41077] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:38049 2023-02-09 14:54:04,523 DEBUG [RS:2;jenkins-hbase4:41077] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-09 14:54:04,528 DEBUG [RS:1;jenkins-hbase4:33553] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9 2023-02-09 14:54:04,528 DEBUG [RS:0;jenkins-hbase4:33423] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9 2023-02-09 14:54:04,528 DEBUG [RS:1;jenkins-hbase4:33553] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:38049 2023-02-09 14:54:04,528 DEBUG [RS:0;jenkins-hbase4:33423] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:38049 2023-02-09 14:54:04,529 DEBUG [RS:1;jenkins-hbase4:33553] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-09 14:54:04,529 DEBUG [RS:0;jenkins-hbase4:33423] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-09 14:54:04,534 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 14:54:04,538 DEBUG [RS:2;jenkins-hbase4:41077] zookeeper.ZKUtil(162): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:04,538 WARN [RS:2;jenkins-hbase4:41077] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 14:54:04,538 INFO [RS:2;jenkins-hbase4:41077] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 14:54:04,538 DEBUG [RS:0;jenkins-hbase4:33423] zookeeper.ZKUtil(162): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:04,538 DEBUG [RS:2;jenkins-hbase4:41077] regionserver.HRegionServer(1947): logDir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/WALs/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:04,539 DEBUG [RS:1;jenkins-hbase4:33553] zookeeper.ZKUtil(162): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:04,539 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,41077,1675954443212] 2023-02-09 14:54:04,539 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,33423,1675954443133] 2023-02-09 14:54:04,539 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,33553,1675954443180] 2023-02-09 14:54:04,538 WARN [RS:0;jenkins-hbase4:33423] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 14:54:04,539 WARN [RS:1;jenkins-hbase4:33553] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-09 14:54:04,541 INFO [RS:0;jenkins-hbase4:33423] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 14:54:04,541 INFO [RS:1;jenkins-hbase4:33553] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 14:54:04,543 DEBUG [RS:0;jenkins-hbase4:33423] regionserver.HRegionServer(1947): logDir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/WALs/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:04,543 DEBUG [RS:1;jenkins-hbase4:33553] regionserver.HRegionServer(1947): logDir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/WALs/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:04,569 DEBUG [RS:2;jenkins-hbase4:41077] zookeeper.ZKUtil(162): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:04,569 DEBUG [RS:1;jenkins-hbase4:33553] zookeeper.ZKUtil(162): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:04,569 DEBUG [RS:0;jenkins-hbase4:33423] zookeeper.ZKUtil(162): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:04,570 DEBUG [RS:2;jenkins-hbase4:41077] zookeeper.ZKUtil(162): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:04,570 DEBUG [RS:0;jenkins-hbase4:33423] zookeeper.ZKUtil(162): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:04,570 DEBUG [RS:1;jenkins-hbase4:33553] zookeeper.ZKUtil(162): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:04,570 DEBUG [RS:2;jenkins-hbase4:41077] zookeeper.ZKUtil(162): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:04,571 DEBUG [RS:0;jenkins-hbase4:33423] zookeeper.ZKUtil(162): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:04,571 DEBUG [RS:1;jenkins-hbase4:33553] zookeeper.ZKUtil(162): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:04,582 DEBUG [RS:2;jenkins-hbase4:41077] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-09 14:54:04,582 DEBUG [RS:1;jenkins-hbase4:33553] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-09 14:54:04,582 DEBUG [RS:0;jenkins-hbase4:33423] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-09 14:54:04,592 INFO [RS:2;jenkins-hbase4:41077] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-09 14:54:04,592 INFO [RS:0;jenkins-hbase4:33423] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-09 14:54:04,592 INFO [RS:1;jenkins-hbase4:33553] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-09 14:54:04,624 INFO [RS:1;jenkins-hbase4:33553] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-09 14:54:04,625 INFO [RS:0;jenkins-hbase4:33423] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-09 14:54:04,624 INFO [RS:2;jenkins-hbase4:41077] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-09 14:54:04,630 INFO [RS:2;jenkins-hbase4:41077] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 14:54:04,630 INFO [RS:1;jenkins-hbase4:33553] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 14:54:04,631 INFO [RS:2;jenkins-hbase4:41077] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:04,630 INFO [RS:0;jenkins-hbase4:33423] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-09 14:54:04,631 INFO [RS:1;jenkins-hbase4:33553] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:04,631 INFO [RS:0;jenkins-hbase4:33423] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:04,633 INFO [RS:1;jenkins-hbase4:33553] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-09 14:54:04,633 INFO [RS:0;jenkins-hbase4:33423] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-09 14:54:04,633 INFO [RS:2;jenkins-hbase4:41077] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-09 14:54:04,642 INFO [RS:0;jenkins-hbase4:33423] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:04,642 INFO [RS:1;jenkins-hbase4:33553] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:04,642 DEBUG [RS:0;jenkins-hbase4:33423] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,642 DEBUG [RS:1;jenkins-hbase4:33553] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,643 INFO [RS:2;jenkins-hbase4:41077] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:04,644 DEBUG [RS:1;jenkins-hbase4:33553] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,643 DEBUG [RS:0;jenkins-hbase4:33423] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,644 DEBUG [RS:1;jenkins-hbase4:33553] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,644 DEBUG [RS:0;jenkins-hbase4:33423] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,644 DEBUG [RS:1;jenkins-hbase4:33553] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,644 DEBUG [RS:0;jenkins-hbase4:33423] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,644 DEBUG [RS:1;jenkins-hbase4:33553] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,644 DEBUG [RS:0;jenkins-hbase4:33423] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,644 DEBUG [RS:1;jenkins-hbase4:33553] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-09 14:54:04,644 DEBUG [RS:0;jenkins-hbase4:33423] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-09 14:54:04,644 DEBUG [RS:2;jenkins-hbase4:41077] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,645 DEBUG [RS:0;jenkins-hbase4:33423] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,645 DEBUG [RS:2;jenkins-hbase4:41077] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,645 DEBUG [RS:0;jenkins-hbase4:33423] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,645 DEBUG [RS:2;jenkins-hbase4:41077] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,645 DEBUG [RS:0;jenkins-hbase4:33423] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,645 DEBUG [RS:2;jenkins-hbase4:41077] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,645 DEBUG [RS:0;jenkins-hbase4:33423] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,645 DEBUG [RS:2;jenkins-hbase4:41077] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,644 DEBUG [RS:1;jenkins-hbase4:33553] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,645 DEBUG [RS:2;jenkins-hbase4:41077] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-09 14:54:04,645 DEBUG [RS:1;jenkins-hbase4:33553] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,645 DEBUG [RS:2;jenkins-hbase4:41077] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,646 DEBUG [RS:2;jenkins-hbase4:41077] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,646 DEBUG [RS:2;jenkins-hbase4:41077] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,646 INFO [RS:0;jenkins-hbase4:33423] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:04,646 DEBUG [RS:2;jenkins-hbase4:41077] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,647 INFO [RS:0;jenkins-hbase4:33423] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:04,647 INFO [RS:0;jenkins-hbase4:33423] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:04,647 DEBUG [RS:1;jenkins-hbase4:33553] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,647 DEBUG [RS:1;jenkins-hbase4:33553] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-09 14:54:04,660 INFO [RS:2;jenkins-hbase4:41077] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:04,660 INFO [RS:2;jenkins-hbase4:41077] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:04,660 INFO [RS:2;jenkins-hbase4:41077] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:04,678 INFO [RS:0;jenkins-hbase4:33423] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-09 14:54:04,680 INFO [RS:1;jenkins-hbase4:33553] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:04,680 INFO [RS:1;jenkins-hbase4:33553] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:04,680 INFO [RS:1;jenkins-hbase4:33553] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:04,680 INFO [RS:0;jenkins-hbase4:33423] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33423,1675954443133-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:04,686 INFO [RS:2;jenkins-hbase4:41077] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-09 14:54:04,688 INFO [RS:2;jenkins-hbase4:41077] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41077,1675954443212-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:04,703 INFO [RS:1;jenkins-hbase4:33553] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-09 14:54:04,703 INFO [RS:1;jenkins-hbase4:33553] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33553,1675954443180-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:04,713 INFO [RS:2;jenkins-hbase4:41077] regionserver.Replication(203): jenkins-hbase4.apache.org,41077,1675954443212 started 2023-02-09 14:54:04,713 INFO [RS:2;jenkins-hbase4:41077] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,41077,1675954443212, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:41077, sessionid=0x1003993358c0003 2023-02-09 14:54:04,715 INFO [RS:0;jenkins-hbase4:33423] regionserver.Replication(203): jenkins-hbase4.apache.org,33423,1675954443133 started 2023-02-09 14:54:04,715 INFO [RS:0;jenkins-hbase4:33423] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,33423,1675954443133, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:33423, sessionid=0x1003993358c0001 2023-02-09 14:54:04,715 DEBUG [RS:2;jenkins-hbase4:41077] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-09 14:54:04,715 DEBUG [RS:0;jenkins-hbase4:33423] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-09 14:54:04,715 DEBUG [RS:2;jenkins-hbase4:41077] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:04,715 DEBUG [RS:0;jenkins-hbase4:33423] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:04,716 DEBUG [RS:2;jenkins-hbase4:41077] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41077,1675954443212' 2023-02-09 14:54:04,716 DEBUG [RS:0;jenkins-hbase4:33423] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33423,1675954443133' 2023-02-09 14:54:04,720 DEBUG [RS:0;jenkins-hbase4:33423] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:04,720 DEBUG [RS:2;jenkins-hbase4:41077] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:04,721 DEBUG [RS:0;jenkins-hbase4:33423] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:04,722 DEBUG [RS:2;jenkins-hbase4:41077] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:04,723 DEBUG [RS:0;jenkins-hbase4:33423] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-09 14:54:04,723 DEBUG [RS:0;jenkins-hbase4:33423] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-09 14:54:04,723 DEBUG [RS:0;jenkins-hbase4:33423] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:04,723 DEBUG [RS:2;jenkins-hbase4:41077] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-09 14:54:04,723 DEBUG [RS:0;jenkins-hbase4:33423] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33423,1675954443133' 2023-02-09 14:54:04,724 DEBUG [RS:2;jenkins-hbase4:41077] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-09 14:54:04,724 DEBUG [RS:0;jenkins-hbase4:33423] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 14:54:04,724 DEBUG [RS:2;jenkins-hbase4:41077] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:04,725 DEBUG [RS:2;jenkins-hbase4:41077] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41077,1675954443212' 2023-02-09 14:54:04,725 DEBUG [RS:2;jenkins-hbase4:41077] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 14:54:04,725 DEBUG [RS:0;jenkins-hbase4:33423] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 14:54:04,726 DEBUG [RS:2;jenkins-hbase4:41077] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 14:54:04,726 DEBUG [RS:0;jenkins-hbase4:33423] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-09 14:54:04,727 INFO [RS:0;jenkins-hbase4:33423] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-09 14:54:04,727 DEBUG [RS:2;jenkins-hbase4:41077] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-09 14:54:04,727 INFO [RS:0;jenkins-hbase4:33423] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-09 14:54:04,727 INFO [RS:2;jenkins-hbase4:41077] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-09 14:54:04,732 INFO [RS:2;jenkins-hbase4:41077] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-09 14:54:04,737 INFO [RS:1;jenkins-hbase4:33553] regionserver.Replication(203): jenkins-hbase4.apache.org,33553,1675954443180 started 2023-02-09 14:54:04,737 INFO [RS:1;jenkins-hbase4:33553] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,33553,1675954443180, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:33553, sessionid=0x1003993358c0002 2023-02-09 14:54:04,737 DEBUG [RS:1;jenkins-hbase4:33553] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-09 14:54:04,737 DEBUG [RS:1;jenkins-hbase4:33553] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:04,737 DEBUG [RS:1;jenkins-hbase4:33553] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33553,1675954443180' 2023-02-09 14:54:04,737 DEBUG [RS:1;jenkins-hbase4:33553] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:04,738 DEBUG [RS:1;jenkins-hbase4:33553] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:04,740 DEBUG [RS:1;jenkins-hbase4:33553] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-09 14:54:04,740 DEBUG [RS:1;jenkins-hbase4:33553] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-09 14:54:04,740 DEBUG [RS:1;jenkins-hbase4:33553] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:04,740 DEBUG [RS:1;jenkins-hbase4:33553] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33553,1675954443180' 2023-02-09 14:54:04,740 DEBUG [RS:1;jenkins-hbase4:33553] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-09 14:54:04,741 DEBUG [RS:1;jenkins-hbase4:33553] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-09 14:54:04,742 DEBUG [RS:1;jenkins-hbase4:33553] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-09 14:54:04,742 INFO [RS:1;jenkins-hbase4:33553] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-09 14:54:04,742 INFO [RS:1;jenkins-hbase4:33553] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-09 14:54:04,839 INFO [RS:2;jenkins-hbase4:41077] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41077%2C1675954443212, suffix=, logDir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/WALs/jenkins-hbase4.apache.org,41077,1675954443212, archiveDir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/oldWALs, maxLogs=32 2023-02-09 14:54:04,839 INFO [RS:0;jenkins-hbase4:33423] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33423%2C1675954443133, suffix=, logDir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/WALs/jenkins-hbase4.apache.org,33423,1675954443133, archiveDir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/oldWALs, maxLogs=32 2023-02-09 14:54:04,844 INFO [RS:1;jenkins-hbase4:33553] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33553%2C1675954443180, suffix=, logDir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/WALs/jenkins-hbase4.apache.org,33553,1675954443180, archiveDir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/oldWALs, maxLogs=32 2023-02-09 14:54:04,866 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38383,DS-0c314c2a-baf0-4782-8322-f37f2df23641,DISK] 2023-02-09 14:54:04,866 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35385,DS-4e02748c-1ef7-4f5d-96f6-080237b6b997,DISK] 2023-02-09 14:54:04,867 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45449,DS-54d025cc-513f-4ba6-87d1-26db3b9f651c,DISK] 2023-02-09 14:54:04,872 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35385,DS-4e02748c-1ef7-4f5d-96f6-080237b6b997,DISK] 2023-02-09 14:54:04,873 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45449,DS-54d025cc-513f-4ba6-87d1-26db3b9f651c,DISK] 2023-02-09 14:54:04,883 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38383,DS-0c314c2a-baf0-4782-8322-f37f2df23641,DISK] 2023-02-09 14:54:04,883 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38383,DS-0c314c2a-baf0-4782-8322-f37f2df23641,DISK] 2023-02-09 14:54:04,883 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35385,DS-4e02748c-1ef7-4f5d-96f6-080237b6b997,DISK] 2023-02-09 14:54:04,884 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45449,DS-54d025cc-513f-4ba6-87d1-26db3b9f651c,DISK] 2023-02-09 14:54:04,893 INFO [RS:2;jenkins-hbase4:41077] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/WALs/jenkins-hbase4.apache.org,41077,1675954443212/jenkins-hbase4.apache.org%2C41077%2C1675954443212.1675954444843 2023-02-09 14:54:04,893 INFO [RS:1;jenkins-hbase4:33553] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/WALs/jenkins-hbase4.apache.org,33553,1675954443180/jenkins-hbase4.apache.org%2C33553%2C1675954443180.1675954444846 2023-02-09 14:54:04,893 DEBUG [RS:2;jenkins-hbase4:41077] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38383,DS-0c314c2a-baf0-4782-8322-f37f2df23641,DISK], DatanodeInfoWithStorage[127.0.0.1:45449,DS-54d025cc-513f-4ba6-87d1-26db3b9f651c,DISK], DatanodeInfoWithStorage[127.0.0.1:35385,DS-4e02748c-1ef7-4f5d-96f6-080237b6b997,DISK]] 2023-02-09 14:54:04,893 DEBUG [RS:1;jenkins-hbase4:33553] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45449,DS-54d025cc-513f-4ba6-87d1-26db3b9f651c,DISK], DatanodeInfoWithStorage[127.0.0.1:38383,DS-0c314c2a-baf0-4782-8322-f37f2df23641,DISK], DatanodeInfoWithStorage[127.0.0.1:35385,DS-4e02748c-1ef7-4f5d-96f6-080237b6b997,DISK]] 2023-02-09 14:54:04,895 INFO [RS:0;jenkins-hbase4:33423] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/WALs/jenkins-hbase4.apache.org,33423,1675954443133/jenkins-hbase4.apache.org%2C33423%2C1675954443133.1675954444843 2023-02-09 14:54:04,895 DEBUG [RS:0;jenkins-hbase4:33423] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35385,DS-4e02748c-1ef7-4f5d-96f6-080237b6b997,DISK], DatanodeInfoWithStorage[127.0.0.1:45449,DS-54d025cc-513f-4ba6-87d1-26db3b9f651c,DISK], DatanodeInfoWithStorage[127.0.0.1:38383,DS-0c314c2a-baf0-4782-8322-f37f2df23641,DISK]] 2023-02-09 14:54:04,922 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:04,925 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-09 14:54:04,927 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740/info 2023-02-09 14:54:04,928 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-09 14:54:04,929 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:04,929 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-09 14:54:04,932 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740/rep_barrier 2023-02-09 14:54:04,932 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-09 14:54:04,933 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:04,933 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-09 14:54:04,935 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740/table 2023-02-09 14:54:04,936 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-09 14:54:04,937 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:04,938 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740 2023-02-09 14:54:04,939 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740 2023-02-09 14:54:04,943 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-09 14:54:04,946 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-09 14:54:04,949 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:54:04,949 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60492401, jitterRate=-0.09859298169612885}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-09 14:54:04,949 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1675954444922Initializing all the Stores at 1675954444924 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675954444924Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954444924Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675954444924Cleaning up temporary data from old regions at 1675954444941 (+17 ms)Cleaning up detritus from prior splits at 1675954444942 (+1 ms)Region opened successfully at 1675954444949 (+7 ms) 2023-02-09 14:54:04,950 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-09 14:54:04,950 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-09 14:54:04,950 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-09 14:54:04,950 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-09 14:54:04,950 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-09 14:54:04,951 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-09 14:54:04,951 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675954444950Disabling compacts and flushes for region at 1675954444950Disabling writes for close at 1675954444950Writing region close event to WAL at 1675954444951 (+1 ms)Closed at 1675954444951 2023-02-09 14:54:04,956 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:04,956 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-09 14:54:04,964 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-09 14:54:04,978 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-09 14:54:04,980 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-09 14:54:05,132 DEBUG [jenkins-hbase4:43829] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-09 14:54:05,136 DEBUG [jenkins-hbase4:43829] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 14:54:05,142 DEBUG [jenkins-hbase4:43829] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 14:54:05,142 DEBUG [jenkins-hbase4:43829] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 14:54:05,142 DEBUG [jenkins-hbase4:43829] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 14:54:05,142 DEBUG [jenkins-hbase4:43829] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 14:54:05,144 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,33423,1675954443133, state=OPENING 2023-02-09 14:54:05,151 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-09 14:54:05,154 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 14:54:05,155 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-09 14:54:05,157 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:54:05,343 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:05,346 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 14:54:05,350 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47018, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 14:54:05,362 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-09 14:54:05,363 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-09 14:54:05,368 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33423%2C1675954443133.meta, suffix=.meta, logDir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/WALs/jenkins-hbase4.apache.org,33423,1675954443133, archiveDir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/oldWALs, maxLogs=32 2023-02-09 14:54:05,392 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38383,DS-0c314c2a-baf0-4782-8322-f37f2df23641,DISK] 2023-02-09 14:54:05,393 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45449,DS-54d025cc-513f-4ba6-87d1-26db3b9f651c,DISK] 2023-02-09 14:54:05,394 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35385,DS-4e02748c-1ef7-4f5d-96f6-080237b6b997,DISK] 2023-02-09 14:54:05,408 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/WALs/jenkins-hbase4.apache.org,33423,1675954443133/jenkins-hbase4.apache.org%2C33423%2C1675954443133.meta.1675954445370.meta 2023-02-09 14:54:05,408 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45449,DS-54d025cc-513f-4ba6-87d1-26db3b9f651c,DISK], DatanodeInfoWithStorage[127.0.0.1:38383,DS-0c314c2a-baf0-4782-8322-f37f2df23641,DISK], DatanodeInfoWithStorage[127.0.0.1:35385,DS-4e02748c-1ef7-4f5d-96f6-080237b6b997,DISK]] 2023-02-09 14:54:05,409 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:54:05,413 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:54:05,414 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-09 14:54:05,435 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-09 14:54:05,438 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-09 14:54:05,442 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-09 14:54:05,443 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:05,443 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-09 14:54:05,443 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-09 14:54:05,446 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-09 14:54:05,447 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740/info 2023-02-09 14:54:05,447 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740/info 2023-02-09 14:54:05,448 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-09 14:54:05,449 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:05,449 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-09 14:54:05,450 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740/rep_barrier 2023-02-09 14:54:05,450 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740/rep_barrier 2023-02-09 14:54:05,451 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-09 14:54:05,452 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:05,452 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-09 14:54:05,453 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740/table 2023-02-09 14:54:05,453 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740/table 2023-02-09 14:54:05,454 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-09 14:54:05,455 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:05,456 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740 2023-02-09 14:54:05,459 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740 2023-02-09 14:54:05,463 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-09 14:54:05,465 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-09 14:54:05,466 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74693716, jitterRate=0.11302310228347778}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-09 14:54:05,466 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1675954445443Writing region info on filesystem at 1675954445443Initializing all the Stores at 1675954445445 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675954445445Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954445445Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675954445445Cleaning up temporary data from old regions at 1675954445461 (+16 ms)Cleaning up detritus from prior splits at 1675954445461Running coprocessor post-open hooks at 1675954445466 (+5 ms)Region opened successfully at 1675954445466 2023-02-09 14:54:05,476 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1675954445335 2023-02-09 14:54:05,493 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-09 14:54:05,494 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-09 14:54:05,494 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,33423,1675954443133, state=OPEN 2023-02-09 14:54:05,497 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-09 14:54:05,497 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-09 14:54:05,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-09 14:54:05,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,33423,1675954443133 in 340 msec 2023-02-09 14:54:05,506 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-09 14:54:05,506 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 538 msec 2023-02-09 14:54:05,511 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 1.3160 sec 2023-02-09 14:54:05,512 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1675954443258, completionTime=-1 2023-02-09 14:54:05,512 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-09 14:54:05,512 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:05,577 DEBUG [hconnection-0x640c2dac-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 14:54:05,580 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47022, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 14:54:05,597 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:05,597 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954505597 2023-02-09 14:54:05,597 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954565597 2023-02-09 14:54:05,598 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 85 msec 2023-02-09 14:54:05,620 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43829,1675954441993-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:05,620 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43829,1675954441993-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:05,620 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43829,1675954441993-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:05,622 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:43829, period=300000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:05,622 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-09 14:54:05,629 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-09 14:54:05,638 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-09 14:54:05,640 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-09 14:54:05,650 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:05,652 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 14:54:05,654 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 14:54:05,675 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/hbase/namespace/0c1f8115bde7013387f9461b40485fe1 2023-02-09 14:54:05,678 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/hbase/namespace/0c1f8115bde7013387f9461b40485fe1 empty. 2023-02-09 14:54:05,678 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/hbase/namespace/0c1f8115bde7013387f9461b40485fe1 2023-02-09 14:54:05,678 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-09 14:54:05,715 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-09 14:54:05,717 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => 0c1f8115bde7013387f9461b40485fe1, NAME => 'hbase:namespace,,1675954445639.0c1f8115bde7013387f9461b40485fe1.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp 2023-02-09 14:54:05,735 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675954445639.0c1f8115bde7013387f9461b40485fe1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:05,735 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing 0c1f8115bde7013387f9461b40485fe1, disabling compactions & flushes 2023-02-09 14:54:05,736 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675954445639.0c1f8115bde7013387f9461b40485fe1. 2023-02-09 14:54:05,736 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675954445639.0c1f8115bde7013387f9461b40485fe1. 2023-02-09 14:54:05,736 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675954445639.0c1f8115bde7013387f9461b40485fe1. after waiting 0 ms 2023-02-09 14:54:05,736 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675954445639.0c1f8115bde7013387f9461b40485fe1. 2023-02-09 14:54:05,736 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675954445639.0c1f8115bde7013387f9461b40485fe1. 2023-02-09 14:54:05,736 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for 0c1f8115bde7013387f9461b40485fe1: Waiting for close lock at 1675954445735Disabling compacts and flushes for region at 1675954445735Disabling writes for close at 1675954445736 (+1 ms)Writing region close event to WAL at 1675954445736Closed at 1675954445736 2023-02-09 14:54:05,741 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 14:54:05,756 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1675954445639.0c1f8115bde7013387f9461b40485fe1.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675954445744"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954445744"}]},"ts":"1675954445744"} 2023-02-09 14:54:05,784 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 14:54:05,786 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 14:54:05,791 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954445786"}]},"ts":"1675954445786"} 2023-02-09 14:54:05,795 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-09 14:54:05,804 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 14:54:05,806 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 14:54:05,806 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 14:54:05,806 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 14:54:05,806 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 14:54:05,809 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=0c1f8115bde7013387f9461b40485fe1, ASSIGN}] 2023-02-09 14:54:05,813 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=0c1f8115bde7013387f9461b40485fe1, ASSIGN 2023-02-09 14:54:05,815 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=0c1f8115bde7013387f9461b40485fe1, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33423,1675954443133; forceNewPlan=false, retain=false 2023-02-09 14:54:05,967 INFO [jenkins-hbase4:43829] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 14:54:05,968 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=0c1f8115bde7013387f9461b40485fe1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:05,969 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1675954445639.0c1f8115bde7013387f9461b40485fe1.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675954445968"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954445968"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954445968"}]},"ts":"1675954445968"} 2023-02-09 14:54:05,973 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 0c1f8115bde7013387f9461b40485fe1, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:54:06,134 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1675954445639.0c1f8115bde7013387f9461b40485fe1. 2023-02-09 14:54:06,135 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0c1f8115bde7013387f9461b40485fe1, NAME => 'hbase:namespace,,1675954445639.0c1f8115bde7013387f9461b40485fe1.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:54:06,136 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:54:06,136 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 0c1f8115bde7013387f9461b40485fe1 2023-02-09 14:54:06,136 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675954445639.0c1f8115bde7013387f9461b40485fe1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:06,136 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 0c1f8115bde7013387f9461b40485fe1 2023-02-09 14:54:06,136 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 0c1f8115bde7013387f9461b40485fe1 2023-02-09 14:54:06,138 INFO [StoreOpener-0c1f8115bde7013387f9461b40485fe1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 0c1f8115bde7013387f9461b40485fe1 2023-02-09 14:54:06,141 DEBUG [StoreOpener-0c1f8115bde7013387f9461b40485fe1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/namespace/0c1f8115bde7013387f9461b40485fe1/info 2023-02-09 14:54:06,141 DEBUG [StoreOpener-0c1f8115bde7013387f9461b40485fe1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/namespace/0c1f8115bde7013387f9461b40485fe1/info 2023-02-09 14:54:06,141 INFO [StoreOpener-0c1f8115bde7013387f9461b40485fe1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0c1f8115bde7013387f9461b40485fe1 columnFamilyName info 2023-02-09 14:54:06,142 INFO [StoreOpener-0c1f8115bde7013387f9461b40485fe1-1] regionserver.HStore(310): Store=0c1f8115bde7013387f9461b40485fe1/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:06,144 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/namespace/0c1f8115bde7013387f9461b40485fe1 2023-02-09 14:54:06,145 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/namespace/0c1f8115bde7013387f9461b40485fe1 2023-02-09 14:54:06,149 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 0c1f8115bde7013387f9461b40485fe1 2023-02-09 14:54:06,152 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/namespace/0c1f8115bde7013387f9461b40485fe1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:54:06,153 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 0c1f8115bde7013387f9461b40485fe1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62152328, jitterRate=-0.07385814189910889}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 14:54:06,153 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 0c1f8115bde7013387f9461b40485fe1: Running coprocessor pre-open hook at 1675954446137Writing region info on filesystem at 1675954446137Initializing all the Stores at 1675954446138 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675954446138Cleaning up temporary data from old regions at 1675954446145 (+7 ms)Cleaning up detritus from prior splits at 1675954446146 (+1 ms)Running coprocessor post-open hooks at 1675954446153 (+7 ms)Region opened successfully at 1675954446153 2023-02-09 14:54:06,155 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1675954445639.0c1f8115bde7013387f9461b40485fe1., pid=6, masterSystemTime=1675954446127 2023-02-09 14:54:06,158 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1675954445639.0c1f8115bde7013387f9461b40485fe1. 2023-02-09 14:54:06,159 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1675954445639.0c1f8115bde7013387f9461b40485fe1. 2023-02-09 14:54:06,160 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:06,161 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1675954445639.0c1f8115bde7013387f9461b40485fe1.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675954446159"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954446159"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954446159"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954446159"}]},"ts":"1675954446159"} 2023-02-09 14:54:06,167 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-09 14:54:06,167 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 0c1f8115bde7013387f9461b40485fe1, server=jenkins-hbase4.apache.org,33423,1675954443133 in 190 msec 2023-02-09 14:54:06,171 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-09 14:54:06,171 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=0c1f8115bde7013387f9461b40485fe1, ASSIGN in 358 msec 2023-02-09 14:54:06,172 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 14:54:06,173 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954446172"}]},"ts":"1675954446172"} 2023-02-09 14:54:06,175 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-09 14:54:06,177 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 14:54:06,180 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 536 msec 2023-02-09 14:54:06,252 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-09 14:54:06,254 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-09 14:54:06,254 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 14:54:06,287 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:06,304 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-09 14:54:06,310 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 29 msec 2023-02-09 14:54:06,320 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:06,331 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-09 14:54:06,336 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 15 msec 2023-02-09 14:54:06,347 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-09 14:54:06,349 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-09 14:54:06,349 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 3.079sec 2023-02-09 14:54:06,352 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-09 14:54:06,353 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-09 14:54:06,353 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-09 14:54:06,355 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43829,1675954441993-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-09 14:54:06,356 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,43829,1675954441993-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-09 14:54:06,365 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-09 14:54:06,441 DEBUG [Listener at localhost/40823] zookeeper.ReadOnlyZKClient(139): Connect 0x37e887de to 127.0.0.1:52509 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-09 14:54:06,448 DEBUG [Listener at localhost/40823] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@c8d5aec, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-09 14:54:06,463 DEBUG [hconnection-0x7e2f5f25-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 14:54:06,479 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47026, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 14:54:06,488 INFO [Listener at localhost/40823] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,43829,1675954441993 2023-02-09 14:54:06,518 INFO [Listener at localhost/40823] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=360, OpenFileDescriptor=688, MaxFileDescriptor=60000, SystemLoadAverage=332, ProcessCount=168, AvailableMemoryMB=3574 2023-02-09 14:54:06,524 WARN [Listener at localhost/40823] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 14:54:06,525 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:06,525 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:06,533 DEBUG [Listener at localhost/40823] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-09 14:54:06,536 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49848, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-09 14:54:06,548 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-02-09 14:54:06,548 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 14:54:06,549 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 14:54:06,555 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:06,557 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 14:54:06,559 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 14:54:06,561 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:06,562 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e empty. 2023-02-09 14:54:06,565 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:06,565 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-09 14:54:06,588 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-02-09 14:54:06,590 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => d88a751f86ea75609ea8bbee5604986e, NAME => 'testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp 2023-02-09 14:54:06,641 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:06,641 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1603): Closing d88a751f86ea75609ea8bbee5604986e, disabling compactions & flushes 2023-02-09 14:54:06,641 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. 2023-02-09 14:54:06,642 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. 2023-02-09 14:54:06,642 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. after waiting 0 ms 2023-02-09 14:54:06,642 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. 2023-02-09 14:54:06,642 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. 2023-02-09 14:54:06,642 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1557): Region close journal for d88a751f86ea75609ea8bbee5604986e: Waiting for close lock at 1675954446641Disabling compacts and flushes for region at 1675954446641Disabling writes for close at 1675954446642 (+1 ms)Writing region close event to WAL at 1675954446642Closed at 1675954446642 2023-02-09 14:54:06,646 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 14:54:06,647 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675954446647"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954446647"}]},"ts":"1675954446647"} 2023-02-09 14:54:06,650 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 14:54:06,651 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 14:54:06,652 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954446651"}]},"ts":"1675954446651"} 2023-02-09 14:54:06,654 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-02-09 14:54:06,658 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 14:54:06,660 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 14:54:06,660 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 14:54:06,660 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 14:54:06,660 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 14:54:06,660 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d88a751f86ea75609ea8bbee5604986e, ASSIGN}] 2023-02-09 14:54:06,663 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d88a751f86ea75609ea8bbee5604986e, ASSIGN 2023-02-09 14:54:06,664 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d88a751f86ea75609ea8bbee5604986e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33423,1675954443133; forceNewPlan=false, retain=false 2023-02-09 14:54:06,815 INFO [jenkins-hbase4:43829] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 14:54:06,816 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=d88a751f86ea75609ea8bbee5604986e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:06,816 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675954446815"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954446815"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954446815"}]},"ts":"1675954446815"} 2023-02-09 14:54:06,819 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure d88a751f86ea75609ea8bbee5604986e, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:54:06,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. 2023-02-09 14:54:06,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d88a751f86ea75609ea8bbee5604986e, NAME => 'testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:54:06,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:54:06,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:06,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:06,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:06,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:06,980 INFO [StoreOpener-d88a751f86ea75609ea8bbee5604986e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:06,982 DEBUG [StoreOpener-d88a751f86ea75609ea8bbee5604986e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1 2023-02-09 14:54:06,983 DEBUG [StoreOpener-d88a751f86ea75609ea8bbee5604986e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1 2023-02-09 14:54:06,983 INFO [StoreOpener-d88a751f86ea75609ea8bbee5604986e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d88a751f86ea75609ea8bbee5604986e columnFamilyName cf1 2023-02-09 14:54:06,984 INFO [StoreOpener-d88a751f86ea75609ea8bbee5604986e-1] regionserver.HStore(310): Store=d88a751f86ea75609ea8bbee5604986e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:06,984 INFO [StoreOpener-d88a751f86ea75609ea8bbee5604986e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:06,986 DEBUG [StoreOpener-d88a751f86ea75609ea8bbee5604986e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2 2023-02-09 14:54:06,986 DEBUG [StoreOpener-d88a751f86ea75609ea8bbee5604986e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2 2023-02-09 14:54:06,986 INFO [StoreOpener-d88a751f86ea75609ea8bbee5604986e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d88a751f86ea75609ea8bbee5604986e columnFamilyName cf2 2023-02-09 14:54:06,987 INFO [StoreOpener-d88a751f86ea75609ea8bbee5604986e-1] regionserver.HStore(310): Store=d88a751f86ea75609ea8bbee5604986e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:06,989 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:06,990 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:06,992 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:54:06,994 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:06,997 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:54:06,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened d88a751f86ea75609ea8bbee5604986e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74046466, jitterRate=0.10337832570075989}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:54:06,998 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for d88a751f86ea75609ea8bbee5604986e: Running coprocessor pre-open hook at 1675954446979Writing region info on filesystem at 1675954446979Initializing all the Stores at 1675954446980 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954446980Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954446980Cleaning up temporary data from old regions at 1675954446991 (+11 ms)Cleaning up detritus from prior splits at 1675954446991Running coprocessor post-open hooks at 1675954446998 (+7 ms)Region opened successfully at 1675954446998 2023-02-09 14:54:06,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e., pid=11, masterSystemTime=1675954446972 2023-02-09 14:54:07,001 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. 2023-02-09 14:54:07,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. 2023-02-09 14:54:07,002 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=d88a751f86ea75609ea8bbee5604986e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,003 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675954447002"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954447002"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954447002"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954447002"}]},"ts":"1675954447002"} 2023-02-09 14:54:07,009 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-09 14:54:07,009 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure d88a751f86ea75609ea8bbee5604986e, server=jenkins-hbase4.apache.org,33423,1675954443133 in 187 msec 2023-02-09 14:54:07,012 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-09 14:54:07,012 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d88a751f86ea75609ea8bbee5604986e, ASSIGN in 349 msec 2023-02-09 14:54:07,013 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 14:54:07,014 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954447013"}]},"ts":"1675954447013"} 2023-02-09 14:54:07,016 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-02-09 14:54:07,018 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 14:54:07,021 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 465 msec 2023-02-09 14:54:07,076 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:07,077 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:07,079 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:07,086 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,087 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-09 14:54:07,087 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:07,088 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-09 14:54:07,088 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-09 14:54:07,088 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,088 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,090 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:07,090 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:07,090 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:07,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:07,090 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:07,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:07,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:07,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:07,090 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:07,090 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:07,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,091 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,091 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,091 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:07,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:07,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,092 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,092 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:07,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,092 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,093 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,093 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,093 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,094 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 14:54:07,094 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:07,094 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 14:54:07,094 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 14:54:07,094 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:07,094 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:07,094 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 14:54:07,095 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 14:54:07,095 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 14:54:07,095 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 14:54:07,095 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 14:54:07,095 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 14:54:07,096 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 14:54:07,096 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:07,096 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:07,096 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. 2023-02-09 14:54:07,096 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. started... 2023-02-09 14:54:07,098 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d88a751f86ea75609ea8bbee5604986e 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 14:54:07,098 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,099 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,099 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,099 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:07,100 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,100 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:07,169 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf1/d45444fdef6c480db9399e5381205e09 2023-02-09 14:54:07,228 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf2/5e92c5580ac042b6b14c9740aa58701f 2023-02-09 14:54:07,237 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf1/d45444fdef6c480db9399e5381205e09 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/d45444fdef6c480db9399e5381205e09 2023-02-09 14:54:07,246 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/d45444fdef6c480db9399e5381205e09, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 14:54:07,249 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf2/5e92c5580ac042b6b14c9740aa58701f as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/5e92c5580ac042b6b14c9740aa58701f 2023-02-09 14:54:07,258 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/5e92c5580ac042b6b14c9740aa58701f, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 14:54:07,260 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for d88a751f86ea75609ea8bbee5604986e in 162ms, sequenceid=6, compaction requested=false 2023-02-09 14:54:07,262 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-02-09 14:54:07,264 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d88a751f86ea75609ea8bbee5604986e: 2023-02-09 14:54:07,264 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. 2023-02-09 14:54:07,264 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:07,264 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:07,265 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,265 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 14:54:07,265 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 14:54:07,266 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,266 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:07,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:07,267 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,267 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:07,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:07,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:07,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:07,270 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-09 14:54:07,270 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-09 14:54:07,270 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@75f9e657[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:07,270 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,274 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,274 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,274 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,274 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 14:54:07,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,274 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,274 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:07,274 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 14:54:07,275 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 14:54:07,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,274 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 14:54:07,275 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 14:54:07,275 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 14:54:07,275 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:07,275 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:07,275 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:07,277 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 14:54:07,277 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:07,277 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 14:54:07,277 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 14:54:07,277 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:07,278 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 14:54:07,278 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:07,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:07,278 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 14:54:07,279 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:07,279 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 14:54:07,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:07,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:07,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:07,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,33423,1675954443133': 2023-02-09 14:54:07,284 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-09 14:54:07,284 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-09 14:54:07,284 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:07,284 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:07,284 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,284 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:07,289 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,290 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,290 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,290 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:07,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:07,290 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,290 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:07,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,290 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:07,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:07,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:07,290 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:07,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:07,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:07,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:07,292 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:07,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:07,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,293 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:07,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,296 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:07,296 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,297 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,306 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,306 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,306 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:07,306 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:07,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:07,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:07,306 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,306 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:07,307 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:07,307 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:07,307 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:07,307 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-09 14:54:07,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:07,306 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:07,307 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:07,307 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,308 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,308 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,308 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,308 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,308 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,308 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:07,307 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:07,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:07,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:07,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:07,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:07,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:07,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:07,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:07,310 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:07,310 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:07,410 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:07,411 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:07,411 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:07,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:07,439 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:07,439 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:07,440 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:07,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:07,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,443 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-09 14:54:07,443 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:07,444 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-09 14:54:07,444 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-09 14:54:07,445 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,445 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,446 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,446 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:07,446 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:07,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:07,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:07,446 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:07,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:07,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:07,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:07,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:07,446 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,446 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:07,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,447 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,447 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,447 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:07,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:07,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:07,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,448 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,448 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,448 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,448 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 14:54:07,448 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,448 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 14:54:07,448 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 14:54:07,448 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,448 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 14:54:07,448 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 14:54:07,448 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,449 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:07,449 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 14:54:07,449 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:07,449 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:07,450 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 14:54:07,450 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 14:54:07,450 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 14:54:07,449 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 14:54:07,451 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 14:54:07,450 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 14:54:07,451 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 14:54:07,451 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:07,451 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:07,451 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. 2023-02-09 14:54:07,451 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. started... 2023-02-09 14:54:07,451 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d88a751f86ea75609ea8bbee5604986e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:54:07,453 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,454 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,454 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,454 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:07,454 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,454 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:07,472 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf1/1ad0534971d044b0a64cdb9f100065dc 2023-02-09 14:54:07,502 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf2/67aa656a77724cce8e350790bcf95e67 2023-02-09 14:54:07,512 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf1/1ad0534971d044b0a64cdb9f100065dc as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/1ad0534971d044b0a64cdb9f100065dc 2023-02-09 14:54:07,521 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/1ad0534971d044b0a64cdb9f100065dc, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 14:54:07,522 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf2/67aa656a77724cce8e350790bcf95e67 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/67aa656a77724cce8e350790bcf95e67 2023-02-09 14:54:07,530 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/67aa656a77724cce8e350790bcf95e67, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 14:54:07,531 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d88a751f86ea75609ea8bbee5604986e in 80ms, sequenceid=19, compaction requested=false 2023-02-09 14:54:07,531 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d88a751f86ea75609ea8bbee5604986e: 2023-02-09 14:54:07,531 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. 2023-02-09 14:54:07,531 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:07,532 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:07,532 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,532 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 14:54:07,532 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 14:54:07,534 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,534 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:07,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:07,534 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,534 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:07,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:07,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:07,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:07,537 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-09 14:54:07,537 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b126033[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:07,537 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-09 14:54:07,537 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,538 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,538 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,538 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,539 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,539 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:07,539 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 14:54:07,539 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 14:54:07,539 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 14:54:07,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,539 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:07,539 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 14:54:07,539 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 14:54:07,539 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 14:54:07,539 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:07,539 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:07,541 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 14:54:07,541 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:07,541 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 14:54:07,543 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 14:54:07,543 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:07,543 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,543 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 14:54:07,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,543 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:07,543 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 14:54:07,544 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 14:54:07,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:07,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:07,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:07,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:07,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:07,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,33423,1675954443133': 2023-02-09 14:54:07,549 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-09 14:54:07,549 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-09 14:54:07,549 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:07,549 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:07,549 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,549 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:07,551 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,551 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,551 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,551 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:07,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:07,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:07,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,551 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:07,551 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,552 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:07,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:07,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,552 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:07,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:07,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:07,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:07,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:07,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:07,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,553 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,554 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:07,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,556 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,556 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,557 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:07,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,567 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,567 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:07,567 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,567 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:07,567 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:07,567 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:07,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:07,567 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:07,567 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:07,567 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,568 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:07,568 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,567 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:07,568 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-09 14:54:07,568 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:07,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:07,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:07,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:07,567 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:07,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:07,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:07,568 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,569 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,569 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:07,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:07,569 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter because node does not exist (not an error) 2023-02-09 14:54:07,569 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:07,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:07,569 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:07,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:07,569 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,570 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,670 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:07,670 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:07,670 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:07,671 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:07,692 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:07,692 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:07,693 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:07,695 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:07,696 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,696 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-09 14:54:07,696 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:07,697 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-09 14:54:07,697 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-09 14:54:07,697 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,697 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,699 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:07,699 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:07,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:07,699 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,699 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:07,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:07,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:07,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:07,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:07,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:07,699 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,699 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:07,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,700 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,700 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,700 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:07,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:07,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:07,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,701 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,701 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,701 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,701 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 14:54:07,701 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 14:54:07,701 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 14:54:07,701 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,701 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,701 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 14:54:07,701 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 14:54:07,701 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:07,701 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,702 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 14:54:07,702 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 14:54:07,702 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:07,701 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:07,703 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 14:54:07,702 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:07,703 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 14:54:07,703 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 14:54:07,703 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. 2023-02-09 14:54:07,703 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 14:54:07,703 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 14:54:07,703 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:07,704 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 14:54:07,703 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. started... 2023-02-09 14:54:07,704 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d88a751f86ea75609ea8bbee5604986e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:54:07,706 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,708 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,708 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,708 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:07,709 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,709 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:07,724 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf1/759ab94148fb4de58c7bebc2199c1c18 2023-02-09 14:54:07,749 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf2/9979bd664e3046c6a8987f27011d84d2 2023-02-09 14:54:07,758 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf1/759ab94148fb4de58c7bebc2199c1c18 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/759ab94148fb4de58c7bebc2199c1c18 2023-02-09 14:54:07,765 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/759ab94148fb4de58c7bebc2199c1c18, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 14:54:07,766 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf2/9979bd664e3046c6a8987f27011d84d2 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/9979bd664e3046c6a8987f27011d84d2 2023-02-09 14:54:07,774 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/9979bd664e3046c6a8987f27011d84d2, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 14:54:07,775 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d88a751f86ea75609ea8bbee5604986e in 71ms, sequenceid=32, compaction requested=true 2023-02-09 14:54:07,775 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d88a751f86ea75609ea8bbee5604986e: 2023-02-09 14:54:07,775 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. 2023-02-09 14:54:07,775 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:07,777 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:07,777 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,777 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 14:54:07,777 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 14:54:07,779 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,779 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:07,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:07,780 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,780 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:07,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:07,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:07,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:07,782 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-09 14:54:07,782 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@398a8829[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:07,782 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-09 14:54:07,782 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,783 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,783 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,783 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,783 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,784 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 14:54:07,784 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 14:54:07,784 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 14:54:07,783 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 14:54:07,784 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 14:54:07,784 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:07,784 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 14:54:07,784 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:07,783 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:07,784 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:07,786 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 14:54:07,786 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:07,786 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 14:54:07,787 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 14:54:07,787 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,787 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 14:54:07,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,787 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:07,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:07,787 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:07,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:07,787 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 14:54:07,788 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 14:54:07,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:07,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:07,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:07,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,33423,1675954443133': 2023-02-09 14:54:07,793 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-09 14:54:07,793 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-09 14:54:07,793 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:07,793 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:07,793 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,793 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:07,795 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,795 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,795 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,795 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,795 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:07,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:07,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:07,795 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:07,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:07,795 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:07,795 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:07,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:07,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:07,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:07,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,796 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,796 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:07,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:07,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,797 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:07,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:07,799 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,799 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,800 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,805 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,805 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:07,805 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,805 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:07,805 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-09 14:54:07,805 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:07,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:07,805 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:07,805 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:07,805 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,805 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:07,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:07,805 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:07,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:07,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:07,805 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,806 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,806 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:07,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:07,806 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,807 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,807 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:07,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:07,806 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:07,806 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:07,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:07,807 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:07,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:07,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:07,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:07,807 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:07,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:07,807 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,908 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:07,908 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:07,908 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:07,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:07,930 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:07,930 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:07,931 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:07,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:07,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,934 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-09 14:54:07,934 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:07,935 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-09 14:54:07,935 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-09 14:54:07,935 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,935 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,936 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:07,936 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:07,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:07,936 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:07,936 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:07,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:07,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:07,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:07,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:07,937 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:07,937 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:07,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,937 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,938 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,938 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:07,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:07,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:07,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,938 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,938 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,938 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,938 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 14:54:07,938 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,938 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 14:54:07,939 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 14:54:07,939 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 14:54:07,939 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,939 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,939 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 14:54:07,939 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 14:54:07,939 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:07,939 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:07,940 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 14:54:07,940 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 14:54:07,939 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:07,940 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 14:54:07,940 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 14:54:07,941 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 14:54:07,941 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:07,941 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. 2023-02-09 14:54:07,941 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 14:54:07,941 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. started... 2023-02-09 14:54:07,941 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:07,941 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 14:54:07,941 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d88a751f86ea75609ea8bbee5604986e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:54:07,943 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,943 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,944 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:07,944 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,945 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:07,945 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:07,958 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf1/4f0c8efcdc0747cc83ffc9e934a1a0ae 2023-02-09 14:54:07,980 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf2/e6c6e6d9cd434d2c81a5a51120ae3cd3 2023-02-09 14:54:07,988 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf1/4f0c8efcdc0747cc83ffc9e934a1a0ae as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/4f0c8efcdc0747cc83ffc9e934a1a0ae 2023-02-09 14:54:07,996 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/4f0c8efcdc0747cc83ffc9e934a1a0ae, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 14:54:07,997 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf2/e6c6e6d9cd434d2c81a5a51120ae3cd3 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/e6c6e6d9cd434d2c81a5a51120ae3cd3 2023-02-09 14:54:08,004 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/e6c6e6d9cd434d2c81a5a51120ae3cd3, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 14:54:08,006 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d88a751f86ea75609ea8bbee5604986e in 65ms, sequenceid=45, compaction requested=true 2023-02-09 14:54:08,006 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d88a751f86ea75609ea8bbee5604986e: 2023-02-09 14:54:08,006 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. 2023-02-09 14:54:08,006 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:08,007 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:08,007 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,007 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 14:54:08,007 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 14:54:08,012 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,012 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:08,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:08,012 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,012 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:08,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:08,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:08,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:08,014 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-09 14:54:08,014 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78a6d44d[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:08,014 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-09 14:54:08,014 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,015 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,015 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,015 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,016 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,016 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 14:54:08,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,016 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 14:54:08,016 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 14:54:08,016 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 14:54:08,016 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 14:54:08,016 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:08,016 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:08,016 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:08,016 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 14:54:08,016 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:08,018 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,018 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 14:54:08,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:08,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:08,018 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:08,018 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 14:54:08,019 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 14:54:08,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:08,019 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 14:54:08,019 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:08,019 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:08,019 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 14:54:08,020 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 14:54:08,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:08,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:08,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,33423,1675954443133': 2023-02-09 14:54:08,024 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-09 14:54:08,024 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-09 14:54:08,024 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:08,024 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:08,024 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,024 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:08,026 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,026 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:08,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:08,026 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,026 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:08,026 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:08,027 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:08,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,027 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:08,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,027 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:08,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:08,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:08,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:08,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:08,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:08,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:08,028 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,028 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:08,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,030 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,031 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,031 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,034 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,034 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,034 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:08,034 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:08,034 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:08,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:08,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:08,034 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:08,034 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-09 14:54:08,034 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:08,034 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:08,034 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:08,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:08,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:08,034 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:08,035 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,035 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:08,035 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:08,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:08,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:08,034 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:08,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:08,035 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,036 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:08,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:08,036 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,036 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:08,036 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:08,036 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,036 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,135 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:08,136 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:08,136 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:08,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:08,157 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:08,157 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:08,158 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:08,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:08,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,160 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-09 14:54:08,160 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:08,161 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-09 14:54:08,161 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-09 14:54:08,161 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,161 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,163 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:08,163 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:08,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:08,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:08,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:08,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:08,163 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,163 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:08,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:08,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:08,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,163 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,163 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:08,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,164 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,164 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,164 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:08,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,164 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,164 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 14:54:08,165 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,165 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 14:54:08,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:08,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:08,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,165 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,165 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:08,165 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,166 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 14:54:08,166 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 14:54:08,166 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 14:54:08,168 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:08,168 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. 2023-02-09 14:54:08,166 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,168 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. started... 2023-02-09 14:54:08,168 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 14:54:08,168 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:08,168 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,168 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d88a751f86ea75609ea8bbee5604986e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:54:08,168 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 14:54:08,168 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:08,168 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:08,169 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 14:54:08,170 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 14:54:08,170 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 14:54:08,170 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 14:54:08,170 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 14:54:08,171 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 14:54:08,172 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,174 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,175 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,175 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:08,175 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,175 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:08,188 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf1/34f2ecd452d84aa78109f161e36d14aa 2023-02-09 14:54:08,213 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf2/8656d86a97d046129d67e2844832bddb 2023-02-09 14:54:08,221 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf1/34f2ecd452d84aa78109f161e36d14aa as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/34f2ecd452d84aa78109f161e36d14aa 2023-02-09 14:54:08,228 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/34f2ecd452d84aa78109f161e36d14aa, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 14:54:08,229 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf2/8656d86a97d046129d67e2844832bddb as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/8656d86a97d046129d67e2844832bddb 2023-02-09 14:54:08,235 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/8656d86a97d046129d67e2844832bddb, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 14:54:08,237 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d88a751f86ea75609ea8bbee5604986e in 69ms, sequenceid=58, compaction requested=true 2023-02-09 14:54:08,237 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d88a751f86ea75609ea8bbee5604986e: 2023-02-09 14:54:08,237 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. 2023-02-09 14:54:08,237 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:08,238 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:08,238 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,238 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 14:54:08,238 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 14:54:08,239 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,239 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:08,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:08,240 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,240 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:08,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:08,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:08,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:08,242 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-09 14:54:08,242 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@42d75e50[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:08,242 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-09 14:54:08,242 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,243 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,243 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,243 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,244 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 14:54:08,244 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 14:54:08,244 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 14:54:08,244 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 14:54:08,244 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 14:54:08,244 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,244 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:08,244 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:08,244 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:08,244 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 14:54:08,244 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:08,245 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,245 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 14:54:08,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:08,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:08,245 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:08,246 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 14:54:08,247 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 14:54:08,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:08,247 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:08,247 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 14:54:08,247 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 14:54:08,248 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:08,248 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 14:54:08,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:08,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:08,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,33423,1675954443133': 2023-02-09 14:54:08,252 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-09 14:54:08,252 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-09 14:54:08,252 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:08,252 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:08,252 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,252 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:08,254 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,254 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,254 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:08,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,254 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,254 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:08,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,254 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:08,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:08,254 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:08,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,255 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:08,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:08,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:08,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:08,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:08,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:08,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:08,255 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,256 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:08,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,257 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:08,258 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,258 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,264 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,264 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:08,264 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:08,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:08,264 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:08,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:08,264 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:08,264 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:08,264 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:08,264 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:08,264 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,264 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:08,264 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-09 14:54:08,265 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:08,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:08,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:08,265 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:08,265 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,265 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,265 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,265 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:08,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:08,265 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,265 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:08,265 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:08,265 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,265 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:08,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:08,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:08,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:08,366 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:08,366 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:08,366 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:08,367 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:08,387 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:08,387 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:08,388 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:08,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:08,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,390 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-09 14:54:08,390 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:08,391 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-09 14:54:08,391 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-09 14:54:08,392 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,392 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,394 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:08,394 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:08,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:08,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:08,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:08,394 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:08,394 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:08,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:08,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:08,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,394 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,395 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:08,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,395 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,395 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,395 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:08,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,395 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:08,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:08,395 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 14:54:08,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,395 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,396 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,396 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,396 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 14:54:08,396 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:08,396 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 14:54:08,396 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 14:54:08,397 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 14:54:08,396 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-09 14:54:08,397 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 14:54:08,397 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,397 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,397 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 14:54:08,397 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-09 14:54:08,397 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:08,397 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:08,397 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 14:54:08,398 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-09 14:54:08,398 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 14:54:08,398 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 14:54:08,398 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:08,398 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:08,398 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. 2023-02-09 14:54:08,398 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. started... 2023-02-09 14:54:08,398 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing d88a751f86ea75609ea8bbee5604986e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:54:08,399 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,400 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,400 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,400 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:08,401 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,401 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:08,415 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf1/49c4412a7cae4fdebc9011209fd82ce0 2023-02-09 14:54:08,437 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf2/98d0e981364c483fa106fc85669f09e3 2023-02-09 14:54:08,446 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf1/49c4412a7cae4fdebc9011209fd82ce0 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/49c4412a7cae4fdebc9011209fd82ce0 2023-02-09 14:54:08,454 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/49c4412a7cae4fdebc9011209fd82ce0, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 14:54:08,455 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf2/98d0e981364c483fa106fc85669f09e3 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/98d0e981364c483fa106fc85669f09e3 2023-02-09 14:54:08,464 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/98d0e981364c483fa106fc85669f09e3, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 14:54:08,467 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d88a751f86ea75609ea8bbee5604986e in 69ms, sequenceid=71, compaction requested=true 2023-02-09 14:54:08,467 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for d88a751f86ea75609ea8bbee5604986e: 2023-02-09 14:54:08,468 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. 2023-02-09 14:54:08,468 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:08,469 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:08,469 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,469 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-09 14:54:08,469 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-09 14:54:08,471 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,471 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:08,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:08,471 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,471 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:08,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:08,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:08,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:08,474 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-09 14:54:08,474 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6a6cbdc1[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:08,474 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-09 14:54:08,474 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,476 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,476 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,476 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,477 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 14:54:08,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,477 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,477 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:08,477 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 14:54:08,477 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 14:54:08,477 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 14:54:08,477 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:08,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,477 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:08,477 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-09 14:54:08,477 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-09 14:54:08,477 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:08,479 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 14:54:08,479 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:08,479 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 14:54:08,480 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,480 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 14:54:08,480 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:08,480 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-09 14:54:08,480 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 14:54:08,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:08,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:08,480 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:08,481 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-09 14:54:08,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:08,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:08,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:08,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,33423,1675954443133': 2023-02-09 14:54:08,485 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-09 14:54:08,485 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-09 14:54:08,485 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:08,485 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:08,485 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,485 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:08,487 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,487 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,487 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:08,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:08,487 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:08,487 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:08,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,487 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,487 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:08,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,487 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:08,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:08,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:08,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:08,488 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:08,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:08,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:08,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:08,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,488 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:08,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,490 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,491 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,491 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,495 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,495 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:08,495 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:08,495 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:08,495 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:08,495 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:08,495 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-09 14:54:08,495 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:08,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:08,495 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:08,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:08,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:08,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:08,495 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:08,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:08,495 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,495 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:08,496 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:08,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:08,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:08,496 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,497 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:08,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:08,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:08,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:08,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:08,497 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,497 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:08,497 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,497 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,497 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:08,497 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,497 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-09 14:54:08,597 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:08,597 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:08,597 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:08,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:08,618 DEBUG [Listener at localhost/40823] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,625 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:08,626 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:08,626 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:08,626 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:08,633 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d88a751f86ea75609ea8bbee5604986e, UNASSIGN}] 2023-02-09 14:54:08,634 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d88a751f86ea75609ea8bbee5604986e, UNASSIGN 2023-02-09 14:54:08,635 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=d88a751f86ea75609ea8bbee5604986e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:08,636 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675954448635"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954448635"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954448635"}]},"ts":"1675954448635"} 2023-02-09 14:54:08,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure d88a751f86ea75609ea8bbee5604986e, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:54:08,797 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:08,797 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing d88a751f86ea75609ea8bbee5604986e, disabling compactions & flushes 2023-02-09 14:54:08,798 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. 2023-02-09 14:54:08,798 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. 2023-02-09 14:54:08,798 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. after waiting 0 ms 2023-02-09 14:54:08,798 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. 2023-02-09 14:54:08,798 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing d88a751f86ea75609ea8bbee5604986e 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 14:54:08,815 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf1/55d51f54deb341209a3cc2ece8cbdd8d 2023-02-09 14:54:08,834 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf2/321c6afd5bbb4d57ab6f35b829265802 2023-02-09 14:54:08,841 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf1/55d51f54deb341209a3cc2ece8cbdd8d as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/55d51f54deb341209a3cc2ece8cbdd8d 2023-02-09 14:54:08,848 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/55d51f54deb341209a3cc2ece8cbdd8d, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 14:54:08,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.tmp/cf2/321c6afd5bbb4d57ab6f35b829265802 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/321c6afd5bbb4d57ab6f35b829265802 2023-02-09 14:54:08,859 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/321c6afd5bbb4d57ab6f35b829265802, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 14:54:08,860 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for d88a751f86ea75609ea8bbee5604986e in 62ms, sequenceid=82, compaction requested=true 2023-02-09 14:54:08,882 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 14:54:08,883 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:54:08,883 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. 2023-02-09 14:54:08,884 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for d88a751f86ea75609ea8bbee5604986e: Waiting for close lock at 1675954448797Running coprocessor pre-close hooks at 1675954448797Disabling compacts and flushes for region at 1675954448797Disabling writes for close at 1675954448798 (+1 ms)Obtaining lock to block concurrent updates at 1675954448798Preparing flush snapshotting stores in d88a751f86ea75609ea8bbee5604986e at 1675954448798Finished memstore snapshotting testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675954448798Flushing stores of testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. at 1675954448799 (+1 ms)Flushing d88a751f86ea75609ea8bbee5604986e/cf1: creating writer at 1675954448799Flushing d88a751f86ea75609ea8bbee5604986e/cf1: appending metadata at 1675954448802 (+3 ms)Flushing d88a751f86ea75609ea8bbee5604986e/cf1: closing flushed file at 1675954448802Flushing d88a751f86ea75609ea8bbee5604986e/cf2: creating writer at 1675954448822 (+20 ms)Flushing d88a751f86ea75609ea8bbee5604986e/cf2: appending metadata at 1675954448824 (+2 ms)Flushing d88a751f86ea75609ea8bbee5604986e/cf2: closing flushed file at 1675954448824Flushing d88a751f86ea75609ea8bbee5604986e/cf1: reopening flushed file at 1675954448842 (+18 ms)Flushing d88a751f86ea75609ea8bbee5604986e/cf2: reopening flushed file at 1675954448854 (+12 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for d88a751f86ea75609ea8bbee5604986e in 62ms, sequenceid=82, compaction requested=true at 1675954448860 (+6 ms)Writing region close event to WAL at 1675954448877 (+17 ms)Running coprocessor post-close hooks at 1675954448883 (+6 ms)Closed at 1675954448883 2023-02-09 14:54:08,886 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:08,887 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=d88a751f86ea75609ea8bbee5604986e, regionState=CLOSED 2023-02-09 14:54:08,888 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675954448887"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954448887"}]},"ts":"1675954448887"} 2023-02-09 14:54:08,892 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-09 14:54:08,892 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure d88a751f86ea75609ea8bbee5604986e, server=jenkins-hbase4.apache.org,33423,1675954443133 in 252 msec 2023-02-09 14:54:08,895 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-02-09 14:54:08,895 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=d88a751f86ea75609ea8bbee5604986e, UNASSIGN in 260 msec 2023-02-09 14:54:08,916 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=d88a751f86ea75609ea8bbee5604986e, threads=14 2023-02-09 14:54:08,917 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/321c6afd5bbb4d57ab6f35b829265802 for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:08,917 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/5e92c5580ac042b6b14c9740aa58701f for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:08,917 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/67aa656a77724cce8e350790bcf95e67 for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:08,917 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/8656d86a97d046129d67e2844832bddb for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:08,917 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/98d0e981364c483fa106fc85669f09e3 for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:08,917 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/9979bd664e3046c6a8987f27011d84d2 for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:08,918 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/e6c6e6d9cd434d2c81a5a51120ae3cd3 for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:08,918 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/1ad0534971d044b0a64cdb9f100065dc for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:08,918 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/34f2ecd452d84aa78109f161e36d14aa for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:08,918 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/49c4412a7cae4fdebc9011209fd82ce0 for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:08,919 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/4f0c8efcdc0747cc83ffc9e934a1a0ae for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:08,919 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/55d51f54deb341209a3cc2ece8cbdd8d for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:08,919 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/759ab94148fb4de58c7bebc2199c1c18 for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:08,920 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/d45444fdef6c480db9399e5381205e09 for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:08,952 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/34f2ecd452d84aa78109f161e36d14aa, top=true 2023-02-09 14:54:08,953 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/49c4412a7cae4fdebc9011209fd82ce0, top=true 2023-02-09 14:54:08,955 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/8656d86a97d046129d67e2844832bddb, top=true 2023-02-09 14:54:08,957 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/321c6afd5bbb4d57ab6f35b829265802, top=true 2023-02-09 14:54:08,958 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/98d0e981364c483fa106fc85669f09e3, top=true 2023-02-09 14:54:08,974 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/55d51f54deb341209a3cc2ece8cbdd8d, top=true 2023-02-09 14:54:09,028 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.splits/e91d34f46fc227e8f7ee0e968792274e/cf2/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-321c6afd5bbb4d57ab6f35b829265802 for child: e91d34f46fc227e8f7ee0e968792274e, parent: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:09,028 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/321c6afd5bbb4d57ab6f35b829265802 for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:09,032 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.splits/e91d34f46fc227e8f7ee0e968792274e/cf2/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-98d0e981364c483fa106fc85669f09e3 for child: e91d34f46fc227e8f7ee0e968792274e, parent: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:09,032 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/98d0e981364c483fa106fc85669f09e3 for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:09,038 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.splits/e91d34f46fc227e8f7ee0e968792274e/cf1/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-55d51f54deb341209a3cc2ece8cbdd8d for child: e91d34f46fc227e8f7ee0e968792274e, parent: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:09,038 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/55d51f54deb341209a3cc2ece8cbdd8d for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:09,038 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.splits/e91d34f46fc227e8f7ee0e968792274e/cf2/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-8656d86a97d046129d67e2844832bddb for child: e91d34f46fc227e8f7ee0e968792274e, parent: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:09,038 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/8656d86a97d046129d67e2844832bddb for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:09,038 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.splits/e91d34f46fc227e8f7ee0e968792274e/cf1/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-49c4412a7cae4fdebc9011209fd82ce0 for child: e91d34f46fc227e8f7ee0e968792274e, parent: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:09,040 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/49c4412a7cae4fdebc9011209fd82ce0 for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:09,041 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/.splits/e91d34f46fc227e8f7ee0e968792274e/cf1/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-34f2ecd452d84aa78109f161e36d14aa for child: e91d34f46fc227e8f7ee0e968792274e, parent: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:09,048 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/34f2ecd452d84aa78109f161e36d14aa for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:09,091 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/1ad0534971d044b0a64cdb9f100065dc for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:09,105 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/e6c6e6d9cd434d2c81a5a51120ae3cd3 for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:09,108 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/5e92c5580ac042b6b14c9740aa58701f for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:09,477 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/4f0c8efcdc0747cc83ffc9e934a1a0ae for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:09,494 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/759ab94148fb4de58c7bebc2199c1c18 for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:09,497 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/67aa656a77724cce8e350790bcf95e67 for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:09,502 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/d45444fdef6c480db9399e5381205e09 for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:09,503 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/9979bd664e3046c6a8987f27011d84d2 for region: d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:09,503 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region d88a751f86ea75609ea8bbee5604986e Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-02-09 14:54:09,553 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 14:54:09,555 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 14:54:09,558 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1675954449558"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1675954449558"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1675954449558"}]},"ts":"1675954449558"} 2023-02-09 14:54:09,559 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675954449558"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954449558"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954449558"}]},"ts":"1675954449558"} 2023-02-09 14:54:09,559 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675954449558"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954449558"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954449558"}]},"ts":"1675954449558"} 2023-02-09 14:54:09,608 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6a1146514f3e0fd1da817878786a40fe, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e91d34f46fc227e8f7ee0e968792274e, ASSIGN}] 2023-02-09 14:54:09,610 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e91d34f46fc227e8f7ee0e968792274e, ASSIGN 2023-02-09 14:54:09,611 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e91d34f46fc227e8f7ee0e968792274e, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33423,1675954443133; forceNewPlan=false, retain=false 2023-02-09 14:54:09,612 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6a1146514f3e0fd1da817878786a40fe, ASSIGN 2023-02-09 14:54:09,613 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6a1146514f3e0fd1da817878786a40fe, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33423,1675954443133; forceNewPlan=false, retain=false 2023-02-09 14:54:09,761 INFO [jenkins-hbase4:43829] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 14:54:09,763 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=6a1146514f3e0fd1da817878786a40fe, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:09,763 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675954449763"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954449763"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954449763"}]},"ts":"1675954449763"} 2023-02-09 14:54:09,765 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 6a1146514f3e0fd1da817878786a40fe, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:54:09,769 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=e91d34f46fc227e8f7ee0e968792274e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:09,769 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675954449769"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954449769"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954449769"}]},"ts":"1675954449769"} 2023-02-09 14:54:09,771 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure e91d34f46fc227e8f7ee0e968792274e, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:54:09,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe. 2023-02-09 14:54:09,922 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6a1146514f3e0fd1da817878786a40fe, NAME => 'testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe.', STARTKEY => '', ENDKEY => '26'} 2023-02-09 14:54:09,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:54:09,923 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 6a1146514f3e0fd1da817878786a40fe 2023-02-09 14:54:09,923 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:09,923 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 6a1146514f3e0fd1da817878786a40fe 2023-02-09 14:54:09,923 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 6a1146514f3e0fd1da817878786a40fe 2023-02-09 14:54:09,925 INFO [StoreOpener-6a1146514f3e0fd1da817878786a40fe-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6a1146514f3e0fd1da817878786a40fe 2023-02-09 14:54:09,927 DEBUG [StoreOpener-6a1146514f3e0fd1da817878786a40fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf1 2023-02-09 14:54:09,927 DEBUG [StoreOpener-6a1146514f3e0fd1da817878786a40fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf1 2023-02-09 14:54:09,928 INFO [StoreOpener-6a1146514f3e0fd1da817878786a40fe-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6a1146514f3e0fd1da817878786a40fe columnFamilyName cf1 2023-02-09 14:54:09,960 DEBUG [StoreOpener-6a1146514f3e0fd1da817878786a40fe-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf1/1ad0534971d044b0a64cdb9f100065dc.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/1ad0534971d044b0a64cdb9f100065dc-bottom 2023-02-09 14:54:09,968 DEBUG [StoreOpener-6a1146514f3e0fd1da817878786a40fe-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf1/4f0c8efcdc0747cc83ffc9e934a1a0ae.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/4f0c8efcdc0747cc83ffc9e934a1a0ae-bottom 2023-02-09 14:54:09,975 DEBUG [StoreOpener-6a1146514f3e0fd1da817878786a40fe-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf1/759ab94148fb4de58c7bebc2199c1c18.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/759ab94148fb4de58c7bebc2199c1c18-bottom 2023-02-09 14:54:09,983 DEBUG [StoreOpener-6a1146514f3e0fd1da817878786a40fe-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf1/d45444fdef6c480db9399e5381205e09.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/d45444fdef6c480db9399e5381205e09-bottom 2023-02-09 14:54:09,984 INFO [StoreOpener-6a1146514f3e0fd1da817878786a40fe-1] regionserver.HStore(310): Store=6a1146514f3e0fd1da817878786a40fe/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:09,984 INFO [StoreOpener-6a1146514f3e0fd1da817878786a40fe-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6a1146514f3e0fd1da817878786a40fe 2023-02-09 14:54:09,986 DEBUG [StoreOpener-6a1146514f3e0fd1da817878786a40fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf2 2023-02-09 14:54:09,986 DEBUG [StoreOpener-6a1146514f3e0fd1da817878786a40fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf2 2023-02-09 14:54:09,986 INFO [StoreOpener-6a1146514f3e0fd1da817878786a40fe-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6a1146514f3e0fd1da817878786a40fe columnFamilyName cf2 2023-02-09 14:54:10,005 DEBUG [StoreOpener-6a1146514f3e0fd1da817878786a40fe-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf2/5e92c5580ac042b6b14c9740aa58701f.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/5e92c5580ac042b6b14c9740aa58701f-bottom 2023-02-09 14:54:10,012 DEBUG [StoreOpener-6a1146514f3e0fd1da817878786a40fe-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf2/67aa656a77724cce8e350790bcf95e67.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/67aa656a77724cce8e350790bcf95e67-bottom 2023-02-09 14:54:10,020 DEBUG [StoreOpener-6a1146514f3e0fd1da817878786a40fe-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf2/9979bd664e3046c6a8987f27011d84d2.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/9979bd664e3046c6a8987f27011d84d2-bottom 2023-02-09 14:54:10,028 DEBUG [StoreOpener-6a1146514f3e0fd1da817878786a40fe-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf2/e6c6e6d9cd434d2c81a5a51120ae3cd3.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/e6c6e6d9cd434d2c81a5a51120ae3cd3-bottom 2023-02-09 14:54:10,028 INFO [StoreOpener-6a1146514f3e0fd1da817878786a40fe-1] regionserver.HStore(310): Store=6a1146514f3e0fd1da817878786a40fe/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:10,029 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe 2023-02-09 14:54:10,032 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe 2023-02-09 14:54:10,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:54:10,036 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 6a1146514f3e0fd1da817878786a40fe 2023-02-09 14:54:10,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 6a1146514f3e0fd1da817878786a40fe; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70697885, jitterRate=0.05348058044910431}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:54:10,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 6a1146514f3e0fd1da817878786a40fe: Running coprocessor pre-open hook at 1675954449924Writing region info on filesystem at 1675954449924Initializing all the Stores at 1675954449925 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954449925Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954449925Cleaning up temporary data from old regions at 1675954450033 (+108 ms)Cleaning up detritus from prior splits at 1675954450033Running coprocessor post-open hooks at 1675954450037 (+4 ms)Region opened successfully at 1675954450037 2023-02-09 14:54:10,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe., pid=17, masterSystemTime=1675954449918 2023-02-09 14:54:10,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:10,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 14:54:10,042 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-09 14:54:10,042 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-09 14:54:10,044 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe. 2023-02-09 14:54:10,044 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe. 2023-02-09 14:54:10,045 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HStore(1912): 6a1146514f3e0fd1da817878786a40fe/cf1 is initiating minor compaction (all files) 2023-02-09 14:54:10,045 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HStore(1912): 6a1146514f3e0fd1da817878786a40fe/cf2 is initiating minor compaction (all files) 2023-02-09 14:54:10,045 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 6a1146514f3e0fd1da817878786a40fe/cf1 in testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe. 2023-02-09 14:54:10,045 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 6a1146514f3e0fd1da817878786a40fe/cf2 in testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe. 2023-02-09 14:54:10,045 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf1/d45444fdef6c480db9399e5381205e09.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/d45444fdef6c480db9399e5381205e09-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf1/1ad0534971d044b0a64cdb9f100065dc.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/1ad0534971d044b0a64cdb9f100065dc-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf1/759ab94148fb4de58c7bebc2199c1c18.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/759ab94148fb4de58c7bebc2199c1c18-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf1/4f0c8efcdc0747cc83ffc9e934a1a0ae.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/4f0c8efcdc0747cc83ffc9e934a1a0ae-bottom] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/.tmp, totalSize=19.9 K 2023-02-09 14:54:10,045 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf2/5e92c5580ac042b6b14c9740aa58701f.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/5e92c5580ac042b6b14c9740aa58701f-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf2/67aa656a77724cce8e350790bcf95e67.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/67aa656a77724cce8e350790bcf95e67-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf2/9979bd664e3046c6a8987f27011d84d2.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/9979bd664e3046c6a8987f27011d84d2-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf2/e6c6e6d9cd434d2c81a5a51120ae3cd3.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/e6c6e6d9cd434d2c81a5a51120ae3cd3-bottom] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/.tmp, totalSize=19.9 K 2023-02-09 14:54:10,046 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=6a1146514f3e0fd1da817878786a40fe, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:10,046 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe. 2023-02-09 14:54:10,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe. 2023-02-09 14:54:10,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e. 2023-02-09 14:54:10,047 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675954450046"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954450046"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954450046"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954450046"}]},"ts":"1675954450046"} 2023-02-09 14:54:10,047 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e91d34f46fc227e8f7ee0e968792274e, NAME => 'testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.', STARTKEY => '26', ENDKEY => ''} 2023-02-09 14:54:10,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:54:10,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:10,048 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting d45444fdef6c480db9399e5381205e09.d88a751f86ea75609ea8bbee5604986e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675954447060 2023-02-09 14:54:10,049 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:10,049 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting 5e92c5580ac042b6b14c9740aa58701f.d88a751f86ea75609ea8bbee5604986e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675954447060 2023-02-09 14:54:10,049 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:10,049 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:10,049 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting 1ad0534971d044b0a64cdb9f100065dc.d88a751f86ea75609ea8bbee5604986e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675954447418 2023-02-09 14:54:10,050 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting 67aa656a77724cce8e350790bcf95e67.d88a751f86ea75609ea8bbee5604986e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675954447418 2023-02-09 14:54:10,050 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting 759ab94148fb4de58c7bebc2199c1c18.d88a751f86ea75609ea8bbee5604986e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675954447673 2023-02-09 14:54:10,051 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting 9979bd664e3046c6a8987f27011d84d2.d88a751f86ea75609ea8bbee5604986e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675954447673 2023-02-09 14:54:10,051 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting 4f0c8efcdc0747cc83ffc9e934a1a0ae.d88a751f86ea75609ea8bbee5604986e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675954447911 2023-02-09 14:54:10,052 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting e6c6e6d9cd434d2c81a5a51120ae3cd3.d88a751f86ea75609ea8bbee5604986e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675954447911 2023-02-09 14:54:10,053 INFO [StoreOpener-e91d34f46fc227e8f7ee0e968792274e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:10,055 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-02-09 14:54:10,055 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 6a1146514f3e0fd1da817878786a40fe, server=jenkins-hbase4.apache.org,33423,1675954443133 in 285 msec 2023-02-09 14:54:10,055 DEBUG [StoreOpener-e91d34f46fc227e8f7ee0e968792274e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1 2023-02-09 14:54:10,055 DEBUG [StoreOpener-e91d34f46fc227e8f7ee0e968792274e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1 2023-02-09 14:54:10,056 INFO [StoreOpener-e91d34f46fc227e8f7ee0e968792274e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e91d34f46fc227e8f7ee0e968792274e columnFamilyName cf1 2023-02-09 14:54:10,058 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6a1146514f3e0fd1da817878786a40fe, ASSIGN in 447 msec 2023-02-09 14:54:10,076 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] throttle.PressureAwareThroughputController(145): 6a1146514f3e0fd1da817878786a40fe#cf2#compaction#14 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:54:10,079 DEBUG [StoreOpener-e91d34f46fc227e8f7ee0e968792274e-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/1ad0534971d044b0a64cdb9f100065dc.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/1ad0534971d044b0a64cdb9f100065dc-top 2023-02-09 14:54:10,080 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6a1146514f3e0fd1da817878786a40fe#cf1#compaction#15 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:54:10,084 DEBUG [StoreOpener-e91d34f46fc227e8f7ee0e968792274e-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/4f0c8efcdc0747cc83ffc9e934a1a0ae.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/4f0c8efcdc0747cc83ffc9e934a1a0ae-top 2023-02-09 14:54:10,091 DEBUG [StoreOpener-e91d34f46fc227e8f7ee0e968792274e-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/759ab94148fb4de58c7bebc2199c1c18.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/759ab94148fb4de58c7bebc2199c1c18-top 2023-02-09 14:54:10,100 DEBUG [StoreOpener-e91d34f46fc227e8f7ee0e968792274e-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/d45444fdef6c480db9399e5381205e09.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/d45444fdef6c480db9399e5381205e09-top 2023-02-09 14:54:10,104 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/.tmp/cf2/20f02f1273a04a88b9cd6df4ab15889a as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf2/20f02f1273a04a88b9cd6df4ab15889a 2023-02-09 14:54:10,109 DEBUG [StoreOpener-e91d34f46fc227e8f7ee0e968792274e-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-34f2ecd452d84aa78109f161e36d14aa 2023-02-09 14:54:10,121 DEBUG [StoreOpener-e91d34f46fc227e8f7ee0e968792274e-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-49c4412a7cae4fdebc9011209fd82ce0 2023-02-09 14:54:10,128 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 6a1146514f3e0fd1da817878786a40fe/cf2 of 6a1146514f3e0fd1da817878786a40fe into 20f02f1273a04a88b9cd6df4ab15889a(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:54:10,130 DEBUG [StoreOpener-e91d34f46fc227e8f7ee0e968792274e-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-55d51f54deb341209a3cc2ece8cbdd8d 2023-02-09 14:54:10,133 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6a1146514f3e0fd1da817878786a40fe: 2023-02-09 14:54:10,133 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe., storeName=6a1146514f3e0fd1da817878786a40fe/cf2, priority=12, startTime=1675954450040; duration=0sec 2023-02-09 14:54:10,133 INFO [StoreOpener-e91d34f46fc227e8f7ee0e968792274e-1] regionserver.HStore(310): Store=e91d34f46fc227e8f7ee0e968792274e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:10,133 INFO [StoreOpener-e91d34f46fc227e8f7ee0e968792274e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:10,134 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:10,134 DEBUG [StoreOpener-e91d34f46fc227e8f7ee0e968792274e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2 2023-02-09 14:54:10,134 DEBUG [StoreOpener-e91d34f46fc227e8f7ee0e968792274e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2 2023-02-09 14:54:10,135 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/.tmp/cf1/0077cc28103441b4b07d4167acf25f51 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf1/0077cc28103441b4b07d4167acf25f51 2023-02-09 14:54:10,135 INFO [StoreOpener-e91d34f46fc227e8f7ee0e968792274e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e91d34f46fc227e8f7ee0e968792274e columnFamilyName cf2 2023-02-09 14:54:10,143 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 6a1146514f3e0fd1da817878786a40fe/cf1 of 6a1146514f3e0fd1da817878786a40fe into 0077cc28103441b4b07d4167acf25f51(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:54:10,145 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6a1146514f3e0fd1da817878786a40fe: 2023-02-09 14:54:10,145 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe., storeName=6a1146514f3e0fd1da817878786a40fe/cf1, priority=12, startTime=1675954450040; duration=0sec 2023-02-09 14:54:10,145 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:10,151 DEBUG [StoreOpener-e91d34f46fc227e8f7ee0e968792274e-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/5e92c5580ac042b6b14c9740aa58701f.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/5e92c5580ac042b6b14c9740aa58701f-top 2023-02-09 14:54:10,157 DEBUG [StoreOpener-e91d34f46fc227e8f7ee0e968792274e-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/67aa656a77724cce8e350790bcf95e67.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/67aa656a77724cce8e350790bcf95e67-top 2023-02-09 14:54:10,165 DEBUG [StoreOpener-e91d34f46fc227e8f7ee0e968792274e-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/9979bd664e3046c6a8987f27011d84d2.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/9979bd664e3046c6a8987f27011d84d2-top 2023-02-09 14:54:10,169 DEBUG [StoreOpener-e91d34f46fc227e8f7ee0e968792274e-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/e6c6e6d9cd434d2c81a5a51120ae3cd3.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/e6c6e6d9cd434d2c81a5a51120ae3cd3-top 2023-02-09 14:54:10,174 DEBUG [StoreOpener-e91d34f46fc227e8f7ee0e968792274e-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-321c6afd5bbb4d57ab6f35b829265802 2023-02-09 14:54:10,178 DEBUG [StoreOpener-e91d34f46fc227e8f7ee0e968792274e-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-8656d86a97d046129d67e2844832bddb 2023-02-09 14:54:10,183 DEBUG [StoreOpener-e91d34f46fc227e8f7ee0e968792274e-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-98d0e981364c483fa106fc85669f09e3 2023-02-09 14:54:10,183 INFO [StoreOpener-e91d34f46fc227e8f7ee0e968792274e-1] regionserver.HStore(310): Store=e91d34f46fc227e8f7ee0e968792274e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:10,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:10,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:10,188 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:54:10,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:10,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e91d34f46fc227e8f7ee0e968792274e; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70410198, jitterRate=0.04919371008872986}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:54:10,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e91d34f46fc227e8f7ee0e968792274e: Running coprocessor pre-open hook at 1675954450049Writing region info on filesystem at 1675954450049Initializing all the Stores at 1675954450051 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954450051Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954450053 (+2 ms)Cleaning up temporary data from old regions at 1675954450186 (+133 ms)Cleaning up detritus from prior splits at 1675954450187 (+1 ms)Running coprocessor post-open hooks at 1675954450190 (+3 ms)Region opened successfully at 1675954450190 2023-02-09 14:54:10,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e., pid=18, masterSystemTime=1675954449918 2023-02-09 14:54:10,192 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:10,193 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 14:54:10,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 14:54:10,193 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 14:54:10,193 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e. 2023-02-09 14:54:10,194 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HStore(1912): e91d34f46fc227e8f7ee0e968792274e/cf1 is initiating minor compaction (all files) 2023-02-09 14:54:10,194 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e. 2023-02-09 14:54:10,194 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of e91d34f46fc227e8f7ee0e968792274e/cf1 in testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e. 2023-02-09 14:54:10,194 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HStore(1912): e91d34f46fc227e8f7ee0e968792274e/cf2 is initiating minor compaction (all files) 2023-02-09 14:54:10,194 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HRegion(2258): Starting compaction of e91d34f46fc227e8f7ee0e968792274e/cf2 in testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e. 2023-02-09 14:54:10,194 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/d45444fdef6c480db9399e5381205e09.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/d45444fdef6c480db9399e5381205e09-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/1ad0534971d044b0a64cdb9f100065dc.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/1ad0534971d044b0a64cdb9f100065dc-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/759ab94148fb4de58c7bebc2199c1c18.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/759ab94148fb4de58c7bebc2199c1c18-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/4f0c8efcdc0747cc83ffc9e934a1a0ae.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/4f0c8efcdc0747cc83ffc9e934a1a0ae-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-34f2ecd452d84aa78109f161e36d14aa, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-49c4412a7cae4fdebc9011209fd82ce0, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-55d51f54deb341209a3cc2ece8cbdd8d] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/.tmp, totalSize=34.9 K 2023-02-09 14:54:10,194 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/5e92c5580ac042b6b14c9740aa58701f.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/5e92c5580ac042b6b14c9740aa58701f-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/67aa656a77724cce8e350790bcf95e67.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/67aa656a77724cce8e350790bcf95e67-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/9979bd664e3046c6a8987f27011d84d2.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/9979bd664e3046c6a8987f27011d84d2-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/e6c6e6d9cd434d2c81a5a51120ae3cd3.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/e6c6e6d9cd434d2c81a5a51120ae3cd3-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-8656d86a97d046129d67e2844832bddb, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-98d0e981364c483fa106fc85669f09e3, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-321c6afd5bbb4d57ab6f35b829265802] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/.tmp, totalSize=34.9 K 2023-02-09 14:54:10,195 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting d45444fdef6c480db9399e5381205e09.d88a751f86ea75609ea8bbee5604986e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675954447060 2023-02-09 14:54:10,197 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting 5e92c5580ac042b6b14c9740aa58701f.d88a751f86ea75609ea8bbee5604986e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675954447060 2023-02-09 14:54:10,197 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting 1ad0534971d044b0a64cdb9f100065dc.d88a751f86ea75609ea8bbee5604986e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675954447418 2023-02-09 14:54:10,198 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting 67aa656a77724cce8e350790bcf95e67.d88a751f86ea75609ea8bbee5604986e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675954447418 2023-02-09 14:54:10,198 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e. 2023-02-09 14:54:10,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e. 2023-02-09 14:54:10,198 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting 759ab94148fb4de58c7bebc2199c1c18.d88a751f86ea75609ea8bbee5604986e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675954447673 2023-02-09 14:54:10,198 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting 9979bd664e3046c6a8987f27011d84d2.d88a751f86ea75609ea8bbee5604986e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675954447673 2023-02-09 14:54:10,199 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting 4f0c8efcdc0747cc83ffc9e934a1a0ae.d88a751f86ea75609ea8bbee5604986e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675954447911 2023-02-09 14:54:10,199 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=e91d34f46fc227e8f7ee0e968792274e, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:10,199 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting e6c6e6d9cd434d2c81a5a51120ae3cd3.d88a751f86ea75609ea8bbee5604986e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675954447911 2023-02-09 14:54:10,199 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675954450199"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954450199"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954450199"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954450199"}]},"ts":"1675954450199"} 2023-02-09 14:54:10,199 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-34f2ecd452d84aa78109f161e36d14aa, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675954448138 2023-02-09 14:54:10,199 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-8656d86a97d046129d67e2844832bddb, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675954448138 2023-02-09 14:54:10,200 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-49c4412a7cae4fdebc9011209fd82ce0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675954448368 2023-02-09 14:54:10,200 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-98d0e981364c483fa106fc85669f09e3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675954448368 2023-02-09 14:54:10,201 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-55d51f54deb341209a3cc2ece8cbdd8d, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675954448599 2023-02-09 14:54:10,201 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-321c6afd5bbb4d57ab6f35b829265802, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675954448599 2023-02-09 14:54:10,206 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=16 2023-02-09 14:54:10,206 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure e91d34f46fc227e8f7ee0e968792274e, server=jenkins-hbase4.apache.org,33423,1675954443133 in 431 msec 2023-02-09 14:54:10,211 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=12 2023-02-09 14:54:10,211 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e91d34f46fc227e8f7ee0e968792274e, ASSIGN in 598 msec 2023-02-09 14:54:10,213 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e in 1.5920 sec 2023-02-09 14:54:10,230 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] throttle.PressureAwareThroughputController(145): e91d34f46fc227e8f7ee0e968792274e#cf2#compaction#17 average throughput is 1.29 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:54:10,230 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] throttle.PressureAwareThroughputController(145): e91d34f46fc227e8f7ee0e968792274e#cf1#compaction#16 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:54:10,246 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/.tmp/cf2/b8beaa600ebe4998a750bb6fe4d0a867 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/b8beaa600ebe4998a750bb6fe4d0a867 2023-02-09 14:54:10,247 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/.tmp/cf1/b1d330bcfc204236b75773177e8e5b87 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/b1d330bcfc204236b75773177e8e5b87 2023-02-09 14:54:10,253 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e91d34f46fc227e8f7ee0e968792274e/cf1 of e91d34f46fc227e8f7ee0e968792274e into b1d330bcfc204236b75773177e8e5b87(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:54:10,253 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e91d34f46fc227e8f7ee0e968792274e/cf2 of e91d34f46fc227e8f7ee0e968792274e into b8beaa600ebe4998a750bb6fe4d0a867(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:54:10,253 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for e91d34f46fc227e8f7ee0e968792274e: 2023-02-09 14:54:10,253 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for e91d34f46fc227e8f7ee0e968792274e: 2023-02-09 14:54:10,253 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e., storeName=e91d34f46fc227e8f7ee0e968792274e/cf1, priority=9, startTime=1675954450192; duration=0sec 2023-02-09 14:54:10,253 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e., storeName=e91d34f46fc227e8f7ee0e968792274e/cf2, priority=9, startTime=1675954450193; duration=0sec 2023-02-09 14:54:10,254 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:10,254 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:10,377 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33423] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:47026 deadline: 1675954510377, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e. is not online on jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:10,509 DEBUG [Listener at localhost/40823] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-09 14:54:10,521 INFO [Listener at localhost/40823] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-09 14:54:10,523 DEBUG [Listener at localhost/40823] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-09 14:54:10,557 WARN [Listener at localhost/40823] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 14:54:10,558 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:10,558 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:10,566 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:10,566 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:10,567 INFO [Listener at localhost/40823] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-02-09 14:54:10,571 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionUnevenDaughter 2023-02-09 14:54:10,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:10,582 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954450582"}]},"ts":"1675954450582"} 2023-02-09 14:54:10,584 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-02-09 14:54:10,586 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:10,586 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:10,586 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-02-09 14:54:10,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 14:54:10,588 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6a1146514f3e0fd1da817878786a40fe, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e91d34f46fc227e8f7ee0e968792274e, UNASSIGN}] 2023-02-09 14:54:10,589 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e91d34f46fc227e8f7ee0e968792274e, UNASSIGN 2023-02-09 14:54:10,590 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=e91d34f46fc227e8f7ee0e968792274e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:10,590 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675954450590"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954450590"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954450590"}]},"ts":"1675954450590"} 2023-02-09 14:54:10,592 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure e91d34f46fc227e8f7ee0e968792274e, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:54:10,595 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6a1146514f3e0fd1da817878786a40fe, UNASSIGN 2023-02-09 14:54:10,595 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=6a1146514f3e0fd1da817878786a40fe, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:10,596 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675954450595"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954450595"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954450595"}]},"ts":"1675954450595"} 2023-02-09 14:54:10,597 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure 6a1146514f3e0fd1da817878786a40fe, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:54:10,691 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:10,691 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:10,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 14:54:10,744 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 6a1146514f3e0fd1da817878786a40fe 2023-02-09 14:54:10,744 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 6a1146514f3e0fd1da817878786a40fe, disabling compactions & flushes 2023-02-09 14:54:10,744 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe. 2023-02-09 14:54:10,744 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe. 2023-02-09 14:54:10,744 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe. after waiting 0 ms 2023-02-09 14:54:10,744 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe. 2023-02-09 14:54:10,750 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf1/d45444fdef6c480db9399e5381205e09.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/d45444fdef6c480db9399e5381205e09-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf1/1ad0534971d044b0a64cdb9f100065dc.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/1ad0534971d044b0a64cdb9f100065dc-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf1/759ab94148fb4de58c7bebc2199c1c18.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/759ab94148fb4de58c7bebc2199c1c18-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf1/4f0c8efcdc0747cc83ffc9e934a1a0ae.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/4f0c8efcdc0747cc83ffc9e934a1a0ae-bottom] to archive 2023-02-09 14:54:10,751 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:10,758 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf1/d45444fdef6c480db9399e5381205e09.d88a751f86ea75609ea8bbee5604986e to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf1/d45444fdef6c480db9399e5381205e09.d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:10,760 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf1/1ad0534971d044b0a64cdb9f100065dc.d88a751f86ea75609ea8bbee5604986e to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf1/1ad0534971d044b0a64cdb9f100065dc.d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:10,762 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf1/759ab94148fb4de58c7bebc2199c1c18.d88a751f86ea75609ea8bbee5604986e to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf1/759ab94148fb4de58c7bebc2199c1c18.d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:10,764 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf1/4f0c8efcdc0747cc83ffc9e934a1a0ae.d88a751f86ea75609ea8bbee5604986e to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf1/4f0c8efcdc0747cc83ffc9e934a1a0ae.d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:10,793 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf2/5e92c5580ac042b6b14c9740aa58701f.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/5e92c5580ac042b6b14c9740aa58701f-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf2/67aa656a77724cce8e350790bcf95e67.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/67aa656a77724cce8e350790bcf95e67-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf2/9979bd664e3046c6a8987f27011d84d2.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/9979bd664e3046c6a8987f27011d84d2-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf2/e6c6e6d9cd434d2c81a5a51120ae3cd3.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/e6c6e6d9cd434d2c81a5a51120ae3cd3-bottom] to archive 2023-02-09 14:54:10,795 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:10,798 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf2/5e92c5580ac042b6b14c9740aa58701f.d88a751f86ea75609ea8bbee5604986e to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf2/5e92c5580ac042b6b14c9740aa58701f.d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:10,801 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf2/67aa656a77724cce8e350790bcf95e67.d88a751f86ea75609ea8bbee5604986e to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf2/67aa656a77724cce8e350790bcf95e67.d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:10,803 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf2/9979bd664e3046c6a8987f27011d84d2.d88a751f86ea75609ea8bbee5604986e to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf2/9979bd664e3046c6a8987f27011d84d2.d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:10,805 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf2/e6c6e6d9cd434d2c81a5a51120ae3cd3.d88a751f86ea75609ea8bbee5604986e to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf2/e6c6e6d9cd434d2c81a5a51120ae3cd3.d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:10,816 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 14:54:10,817 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:54:10,817 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe. 2023-02-09 14:54:10,817 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 6a1146514f3e0fd1da817878786a40fe: Waiting for close lock at 1675954450744Running coprocessor pre-close hooks at 1675954450744Disabling compacts and flushes for region at 1675954450744Disabling writes for close at 1675954450744Writing region close event to WAL at 1675954450807 (+63 ms)Running coprocessor post-close hooks at 1675954450817 (+10 ms)Closed at 1675954450817 2023-02-09 14:54:10,819 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 6a1146514f3e0fd1da817878786a40fe 2023-02-09 14:54:10,820 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:10,820 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e91d34f46fc227e8f7ee0e968792274e, disabling compactions & flushes 2023-02-09 14:54:10,820 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e. 2023-02-09 14:54:10,820 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e. 2023-02-09 14:54:10,820 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e. after waiting 0 ms 2023-02-09 14:54:10,820 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e. 2023-02-09 14:54:10,820 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=6a1146514f3e0fd1da817878786a40fe, regionState=CLOSED 2023-02-09 14:54:10,821 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675954450820"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954450820"}]},"ts":"1675954450820"} 2023-02-09 14:54:10,825 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-02-09 14:54:10,825 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure 6a1146514f3e0fd1da817878786a40fe, server=jenkins-hbase4.apache.org,33423,1675954443133 in 225 msec 2023-02-09 14:54:10,826 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=6a1146514f3e0fd1da817878786a40fe, UNASSIGN in 237 msec 2023-02-09 14:54:10,829 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/d45444fdef6c480db9399e5381205e09.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/d45444fdef6c480db9399e5381205e09-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/1ad0534971d044b0a64cdb9f100065dc.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/1ad0534971d044b0a64cdb9f100065dc-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/759ab94148fb4de58c7bebc2199c1c18.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/759ab94148fb4de58c7bebc2199c1c18-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/4f0c8efcdc0747cc83ffc9e934a1a0ae.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/4f0c8efcdc0747cc83ffc9e934a1a0ae-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-34f2ecd452d84aa78109f161e36d14aa, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-49c4412a7cae4fdebc9011209fd82ce0, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-55d51f54deb341209a3cc2ece8cbdd8d] to archive 2023-02-09 14:54:10,830 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:10,832 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/d45444fdef6c480db9399e5381205e09.d88a751f86ea75609ea8bbee5604986e to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/d45444fdef6c480db9399e5381205e09.d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:10,834 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/1ad0534971d044b0a64cdb9f100065dc.d88a751f86ea75609ea8bbee5604986e to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/1ad0534971d044b0a64cdb9f100065dc.d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:10,835 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/759ab94148fb4de58c7bebc2199c1c18.d88a751f86ea75609ea8bbee5604986e to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/759ab94148fb4de58c7bebc2199c1c18.d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:10,837 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/4f0c8efcdc0747cc83ffc9e934a1a0ae.d88a751f86ea75609ea8bbee5604986e to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/4f0c8efcdc0747cc83ffc9e934a1a0ae.d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:10,838 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-34f2ecd452d84aa78109f161e36d14aa to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-34f2ecd452d84aa78109f161e36d14aa 2023-02-09 14:54:10,840 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-49c4412a7cae4fdebc9011209fd82ce0 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-49c4412a7cae4fdebc9011209fd82ce0 2023-02-09 14:54:10,841 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-55d51f54deb341209a3cc2ece8cbdd8d to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-55d51f54deb341209a3cc2ece8cbdd8d 2023-02-09 14:54:10,843 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/5e92c5580ac042b6b14c9740aa58701f.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/5e92c5580ac042b6b14c9740aa58701f-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/67aa656a77724cce8e350790bcf95e67.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/67aa656a77724cce8e350790bcf95e67-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/9979bd664e3046c6a8987f27011d84d2.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/9979bd664e3046c6a8987f27011d84d2-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/e6c6e6d9cd434d2c81a5a51120ae3cd3.d88a751f86ea75609ea8bbee5604986e->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/e6c6e6d9cd434d2c81a5a51120ae3cd3-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-8656d86a97d046129d67e2844832bddb, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-98d0e981364c483fa106fc85669f09e3, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-321c6afd5bbb4d57ab6f35b829265802] to archive 2023-02-09 14:54:10,844 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:10,846 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/5e92c5580ac042b6b14c9740aa58701f.d88a751f86ea75609ea8bbee5604986e to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/5e92c5580ac042b6b14c9740aa58701f.d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:10,848 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/67aa656a77724cce8e350790bcf95e67.d88a751f86ea75609ea8bbee5604986e to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/67aa656a77724cce8e350790bcf95e67.d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:10,849 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/9979bd664e3046c6a8987f27011d84d2.d88a751f86ea75609ea8bbee5604986e to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/9979bd664e3046c6a8987f27011d84d2.d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:10,851 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/e6c6e6d9cd434d2c81a5a51120ae3cd3.d88a751f86ea75609ea8bbee5604986e to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/e6c6e6d9cd434d2c81a5a51120ae3cd3.d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:10,852 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-8656d86a97d046129d67e2844832bddb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-8656d86a97d046129d67e2844832bddb 2023-02-09 14:54:10,854 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-98d0e981364c483fa106fc85669f09e3 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-98d0e981364c483fa106fc85669f09e3 2023-02-09 14:54:10,855 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-321c6afd5bbb4d57ab6f35b829265802 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/testSplitTableRegionUnevenDaughter=d88a751f86ea75609ea8bbee5604986e-321c6afd5bbb4d57ab6f35b829265802 2023-02-09 14:54:10,862 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 14:54:10,863 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:54:10,864 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e. 2023-02-09 14:54:10,864 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e91d34f46fc227e8f7ee0e968792274e: Waiting for close lock at 1675954450820Running coprocessor pre-close hooks at 1675954450820Disabling compacts and flushes for region at 1675954450820Disabling writes for close at 1675954450820Writing region close event to WAL at 1675954450856 (+36 ms)Running coprocessor post-close hooks at 1675954450863 (+7 ms)Closed at 1675954450864 (+1 ms) 2023-02-09 14:54:10,866 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:10,867 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=e91d34f46fc227e8f7ee0e968792274e, regionState=CLOSED 2023-02-09 14:54:10,867 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675954450867"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954450867"}]},"ts":"1675954450867"} 2023-02-09 14:54:10,872 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-02-09 14:54:10,872 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure e91d34f46fc227e8f7ee0e968792274e, server=jenkins-hbase4.apache.org,33423,1675954443133 in 277 msec 2023-02-09 14:54:10,874 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=19 2023-02-09 14:54:10,874 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=e91d34f46fc227e8f7ee0e968792274e, UNASSIGN in 284 msec 2023-02-09 14:54:10,876 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954450876"}]},"ts":"1675954450876"} 2023-02-09 14:54:10,878 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-02-09 14:54:10,880 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-02-09 14:54:10,883 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 308 msec 2023-02-09 14:54:10,892 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:10,893 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:10,894 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-09 14:54:10,894 INFO [Listener at localhost/40823] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-02-09 14:54:10,895 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:10,895 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:10,899 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionUnevenDaughter 2023-02-09 14:54:10,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:10,909 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:10,911 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:10,911 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:10,911 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:10,913 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-09 14:54:10,919 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:10,919 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe 2023-02-09 14:54:10,919 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:10,922 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/recovered.edits] 2023-02-09 14:54:10,922 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/recovered.edits] 2023-02-09 14:54:10,922 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf1, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf2, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/recovered.edits] 2023-02-09 14:54:10,927 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/b1d330bcfc204236b75773177e8e5b87 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf1/b1d330bcfc204236b75773177e8e5b87 2023-02-09 14:54:10,927 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/1ad0534971d044b0a64cdb9f100065dc to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/1ad0534971d044b0a64cdb9f100065dc 2023-02-09 14:54:10,927 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf1/0077cc28103441b4b07d4167acf25f51 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf1/0077cc28103441b4b07d4167acf25f51 2023-02-09 14:54:10,929 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/34f2ecd452d84aa78109f161e36d14aa to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/34f2ecd452d84aa78109f161e36d14aa 2023-02-09 14:54:10,930 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/b8beaa600ebe4998a750bb6fe4d0a867 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/cf2/b8beaa600ebe4998a750bb6fe4d0a867 2023-02-09 14:54:10,930 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/49c4412a7cae4fdebc9011209fd82ce0 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/49c4412a7cae4fdebc9011209fd82ce0 2023-02-09 14:54:10,931 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf2/20f02f1273a04a88b9cd6df4ab15889a to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/cf2/20f02f1273a04a88b9cd6df4ab15889a 2023-02-09 14:54:10,933 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/4f0c8efcdc0747cc83ffc9e934a1a0ae to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/4f0c8efcdc0747cc83ffc9e934a1a0ae 2023-02-09 14:54:10,935 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/55d51f54deb341209a3cc2ece8cbdd8d to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/55d51f54deb341209a3cc2ece8cbdd8d 2023-02-09 14:54:10,935 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/recovered.edits/90.seqid to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe/recovered.edits/90.seqid 2023-02-09 14:54:10,935 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/recovered.edits/90.seqid to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e/recovered.edits/90.seqid 2023-02-09 14:54:10,937 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/6a1146514f3e0fd1da817878786a40fe 2023-02-09 14:54:10,937 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:10,938 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/759ab94148fb4de58c7bebc2199c1c18 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/759ab94148fb4de58c7bebc2199c1c18 2023-02-09 14:54:10,939 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/d45444fdef6c480db9399e5381205e09 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf1/d45444fdef6c480db9399e5381205e09 2023-02-09 14:54:10,942 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/321c6afd5bbb4d57ab6f35b829265802 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/321c6afd5bbb4d57ab6f35b829265802 2023-02-09 14:54:10,944 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/5e92c5580ac042b6b14c9740aa58701f to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/5e92c5580ac042b6b14c9740aa58701f 2023-02-09 14:54:10,946 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/67aa656a77724cce8e350790bcf95e67 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/67aa656a77724cce8e350790bcf95e67 2023-02-09 14:54:10,948 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/8656d86a97d046129d67e2844832bddb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/8656d86a97d046129d67e2844832bddb 2023-02-09 14:54:10,949 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/98d0e981364c483fa106fc85669f09e3 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/98d0e981364c483fa106fc85669f09e3 2023-02-09 14:54:10,950 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/9979bd664e3046c6a8987f27011d84d2 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/9979bd664e3046c6a8987f27011d84d2 2023-02-09 14:54:10,952 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/e6c6e6d9cd434d2c81a5a51120ae3cd3 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/cf2/e6c6e6d9cd434d2c81a5a51120ae3cd3 2023-02-09 14:54:10,956 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/recovered.edits/85.seqid to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e/recovered.edits/85.seqid 2023-02-09 14:54:10,956 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionUnevenDaughter/d88a751f86ea75609ea8bbee5604986e 2023-02-09 14:54:10,956 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-09 14:54:10,959 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:10,972 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 14:54:10,982 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-02-09 14:54:11,013 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:11,013 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:11,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-09 14:54:11,026 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-02-09 14:54:11,028 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:11,028 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-02-09 14:54:11,028 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954451028"}]},"ts":"9223372036854775807"} 2023-02-09 14:54:11,029 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954451028"}]},"ts":"9223372036854775807"} 2023-02-09 14:54:11,029 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954451028"}]},"ts":"9223372036854775807"} 2023-02-09 14:54:11,032 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 14:54:11,032 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => d88a751f86ea75609ea8bbee5604986e, NAME => 'testSplitTableRegionUnevenDaughter,,1675954446553.d88a751f86ea75609ea8bbee5604986e.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 6a1146514f3e0fd1da817878786a40fe, NAME => 'testSplitTableRegionUnevenDaughter,,1675954448619.6a1146514f3e0fd1da817878786a40fe.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => e91d34f46fc227e8f7ee0e968792274e, NAME => 'testSplitTableRegionUnevenDaughter,26,1675954448619.e91d34f46fc227e8f7ee0e968792274e.', STARTKEY => '26', ENDKEY => ''}] 2023-02-09 14:54:11,032 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-02-09 14:54:11,032 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675954451032"}]},"ts":"9223372036854775807"} 2023-02-09 14:54:11,034 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-02-09 14:54:11,036 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:11,040 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 137 msec 2023-02-09 14:54:11,043 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-09 14:54:11,044 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-09 14:54:11,215 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:11,215 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:11,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-09 14:54:11,216 INFO [Listener at localhost/40823] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-02-09 14:54:11,235 INFO [Listener at localhost/40823] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=392 (was 360) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1922078276_15 at /127.0.0.1:54468 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1922078276_15 at /127.0.0.1:54498 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1922078276_15 at /127.0.0.1:54430 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x640c2dac-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1922078276_15 at /127.0.0.1:37024 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x640c2dac-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x640c2dac-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x640c2dac-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1922078276_15 at /127.0.0.1:37054 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1922078276_15 at /127.0.0.1:36712 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1922078276_15 at /127.0.0.1:54438 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1922078276_15 at /127.0.0.1:37020 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:33423-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1922078276_15 at /127.0.0.1:54412 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1922078276_15 at /127.0.0.1:36698 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1922078276_15 at /127.0.0.1:54416 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1922078276_15 at /127.0.0.1:36664 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1922078276_15 at /127.0.0.1:37030 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1922078276_15 at /127.0.0.1:54452 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1922078276_15 at /127.0.0.1:54390 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1922078276_15 at /127.0.0.1:54372 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=844 (was 688) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=305 (was 332), ProcessCount=168 (was 168), AvailableMemoryMB=3443 (was 3574) 2023-02-09 14:54:11,251 INFO [Listener at localhost/40823] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=392, OpenFileDescriptor=844, MaxFileDescriptor=60000, SystemLoadAverage=305, ProcessCount=168, AvailableMemoryMB=3442 2023-02-09 14:54:11,251 WARN [Listener at localhost/40823] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 14:54:11,252 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:11,252 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:11,257 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 14:54:11,258 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 14:54:11,260 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:11,266 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 14:54:11,267 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 14:54:11,269 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:11,269 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff empty. 2023-02-09 14:54:11,270 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:11,270 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-09 14:54:11,282 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-02-09 14:54:11,283 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7671): creating {ENCODED => 2f41522f4f390d8b073e3a5237c42fff, NAME => 'testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp 2023-02-09 14:54:11,294 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:11,294 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1603): Closing 2f41522f4f390d8b073e3a5237c42fff, disabling compactions & flushes 2023-02-09 14:54:11,294 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. 2023-02-09 14:54:11,294 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. 2023-02-09 14:54:11,294 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. after waiting 0 ms 2023-02-09 14:54:11,294 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. 2023-02-09 14:54:11,294 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. 2023-02-09 14:54:11,294 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1557): Region close journal for 2f41522f4f390d8b073e3a5237c42fff: Waiting for close lock at 1675954451294Disabling compacts and flushes for region at 1675954451294Disabling writes for close at 1675954451294Writing region close event to WAL at 1675954451294Closed at 1675954451294 2023-02-09 14:54:11,297 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 14:54:11,298 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675954451298"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954451298"}]},"ts":"1675954451298"} 2023-02-09 14:54:11,300 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 14:54:11,301 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 14:54:11,301 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954451301"}]},"ts":"1675954451301"} 2023-02-09 14:54:11,303 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-02-09 14:54:11,305 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 14:54:11,307 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 14:54:11,307 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 14:54:11,307 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 14:54:11,307 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 14:54:11,307 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, ASSIGN}] 2023-02-09 14:54:11,309 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, ASSIGN 2023-02-09 14:54:11,310 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41077,1675954443212; forceNewPlan=false, retain=false 2023-02-09 14:54:11,460 INFO [jenkins-hbase4:43829] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 14:54:11,461 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=2f41522f4f390d8b073e3a5237c42fff, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:11,462 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675954451461"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954451461"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954451461"}]},"ts":"1675954451461"} 2023-02-09 14:54:11,464 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 2f41522f4f390d8b073e3a5237c42fff, server=jenkins-hbase4.apache.org,41077,1675954443212}] 2023-02-09 14:54:11,617 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:11,617 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 14:54:11,621 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55214, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 14:54:11,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. 2023-02-09 14:54:11,625 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2f41522f4f390d8b073e3a5237c42fff, NAME => 'testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:54:11,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:54:11,625 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:11,625 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:11,625 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:11,625 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:11,626 INFO [StoreOpener-2f41522f4f390d8b073e3a5237c42fff-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:11,628 DEBUG [StoreOpener-2f41522f4f390d8b073e3a5237c42fff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1 2023-02-09 14:54:11,628 DEBUG [StoreOpener-2f41522f4f390d8b073e3a5237c42fff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1 2023-02-09 14:54:11,628 INFO [StoreOpener-2f41522f4f390d8b073e3a5237c42fff-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2f41522f4f390d8b073e3a5237c42fff columnFamilyName cf1 2023-02-09 14:54:11,629 INFO [StoreOpener-2f41522f4f390d8b073e3a5237c42fff-1] regionserver.HStore(310): Store=2f41522f4f390d8b073e3a5237c42fff/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:11,629 INFO [StoreOpener-2f41522f4f390d8b073e3a5237c42fff-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:11,631 DEBUG [StoreOpener-2f41522f4f390d8b073e3a5237c42fff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2 2023-02-09 14:54:11,631 DEBUG [StoreOpener-2f41522f4f390d8b073e3a5237c42fff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2 2023-02-09 14:54:11,632 INFO [StoreOpener-2f41522f4f390d8b073e3a5237c42fff-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2f41522f4f390d8b073e3a5237c42fff columnFamilyName cf2 2023-02-09 14:54:11,633 INFO [StoreOpener-2f41522f4f390d8b073e3a5237c42fff-1] regionserver.HStore(310): Store=2f41522f4f390d8b073e3a5237c42fff/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:11,634 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:11,634 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:11,636 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:54:11,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:11,641 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:54:11,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 2f41522f4f390d8b073e3a5237c42fff; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75392593, jitterRate=0.12343718111515045}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:54:11,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 2f41522f4f390d8b073e3a5237c42fff: Running coprocessor pre-open hook at 1675954451625Writing region info on filesystem at 1675954451625Initializing all the Stores at 1675954451626 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954451626Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954451626Cleaning up temporary data from old regions at 1675954451635 (+9 ms)Cleaning up detritus from prior splits at 1675954451635Running coprocessor post-open hooks at 1675954451642 (+7 ms)Region opened successfully at 1675954451642 2023-02-09 14:54:11,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff., pid=27, masterSystemTime=1675954451617 2023-02-09 14:54:11,647 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. 2023-02-09 14:54:11,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. 2023-02-09 14:54:11,648 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=2f41522f4f390d8b073e3a5237c42fff, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:11,648 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675954451648"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954451648"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954451648"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954451648"}]},"ts":"1675954451648"} 2023-02-09 14:54:11,653 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-02-09 14:54:11,653 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 2f41522f4f390d8b073e3a5237c42fff, server=jenkins-hbase4.apache.org,41077,1675954443212 in 186 msec 2023-02-09 14:54:11,655 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-02-09 14:54:11,655 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, ASSIGN in 346 msec 2023-02-09 14:54:11,656 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 14:54:11,656 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954451656"}]},"ts":"1675954451656"} 2023-02-09 14:54:11,658 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-02-09 14:54:11,661 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 14:54:11,662 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 403 msec 2023-02-09 14:54:11,763 DEBUG [Listener at localhost/40823] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 14:54:11,767 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55216, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 14:54:11,774 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:11,774 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:11,775 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:11,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-09 14:54:11,777 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-09 14:54:11,777 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:11,777 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-09 14:54:11,778 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-09 14:54:11,778 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:11,778 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:11,780 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:11,780 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:11,780 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:11,780 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:11,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:11,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:11,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:11,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:11,781 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:11,781 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:11,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:11,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:11,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:11,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:11,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:11,781 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:11,781 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:11,781 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:11,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:11,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:11,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:11,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:11,782 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 14:54:11,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:11,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:11,782 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 14:54:11,782 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 14:54:11,782 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 14:54:11,782 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 14:54:11,782 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:11,782 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 14:54:11,782 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 14:54:11,783 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:11,783 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 14:54:11,783 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 14:54:11,783 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 14:54:11,784 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:11,783 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 14:54:11,784 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:11,784 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:11,784 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. 2023-02-09 14:54:11,784 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 14:54:11,784 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 14:54:11,784 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 14:54:11,784 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. started... 2023-02-09 14:54:11,784 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 14:54:11,784 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 14:54:11,785 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2f41522f4f390d8b073e3a5237c42fff 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 14:54:11,786 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:11,787 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:11,787 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:11,787 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:11,788 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:11,788 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:11,798 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf1/24d455dade92499595cdab11694c97e3 2023-02-09 14:54:11,815 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d 2023-02-09 14:54:11,820 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf1/24d455dade92499595cdab11694c97e3 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/24d455dade92499595cdab11694c97e3 2023-02-09 14:54:11,825 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/24d455dade92499595cdab11694c97e3, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 14:54:11,826 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d 2023-02-09 14:54:11,830 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 14:54:11,831 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 2f41522f4f390d8b073e3a5237c42fff in 46ms, sequenceid=6, compaction requested=false 2023-02-09 14:54:11,831 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-02-09 14:54:11,832 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2f41522f4f390d8b073e3a5237c42fff: 2023-02-09 14:54:11,832 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. 2023-02-09 14:54:11,832 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:11,832 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:11,832 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:11,832 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 14:54:11,832 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 14:54:11,834 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:11,834 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:11,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:11,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:11,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:11,834 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:11,834 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:11,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:11,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:11,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:11,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:11,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:11,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:11,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:11,836 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-09 14:54:11,836 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7caab65[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:11,836 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-09 14:54:11,836 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:11,838 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:11,838 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:11,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:11,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:11,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:11,838 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:11,838 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 14:54:11,838 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:11,838 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 14:54:11,838 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:11,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:11,839 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:11,839 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 14:54:11,839 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 14:54:11,839 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:11,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:11,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:11,839 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 14:54:11,839 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 14:54:11,839 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:11,840 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 14:54:11,840 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 14:54:11,840 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:11,840 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:11,840 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 14:54:11,841 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 14:54:11,842 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:11,842 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 14:54:11,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:11,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:11,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:11,842 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:11,842 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 14:54:11,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:11,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:11,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:11,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:11,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:11,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:11,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:11,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:11,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:11,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:11,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:11,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,41077,1675954443212': 2023-02-09 14:54:11,845 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-09 14:54:11,845 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-09 14:54:11,845 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:11,845 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:11,845 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-09 14:54:11,845 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:11,847 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:11,847 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:11,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:11,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:11,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:11,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:11,847 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:11,847 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:11,847 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:11,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:11,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:11,847 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:11,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:11,847 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:11,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:11,848 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:11,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:11,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:11,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:11,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:11,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:11,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:11,848 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:11,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:11,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:11,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:11,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:11,849 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:11,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:11,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:11,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:11,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:11,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:11,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:11,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:11,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:11,850 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:11,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:11,851 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:11,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:11,851 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:11,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:11,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:11,855 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:11,855 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:11,855 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:11,855 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:11,855 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:11,855 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:11,855 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:11,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:11,855 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:11,855 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:11,855 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-09 14:54:11,856 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:11,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:11,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:11,855 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:11,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:11,855 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:11,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:11,856 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:11,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:11,856 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:11,856 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:11,856 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:11,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:11,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:11,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:11,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:11,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:11,856 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:11,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:11,857 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:11,857 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:11,857 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:11,857 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:11,956 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:11,957 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:11,957 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:11,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:11,975 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:11,975 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:11,976 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:11,977 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:11,977 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-09 14:54:11,977 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-09 14:54:11,978 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:11,978 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-09 14:54:11,978 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-09 14:54:11,979 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:11,979 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:11,980 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:11,980 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:11,980 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:11,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:11,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:11,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:11,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:11,980 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:11,980 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:11,980 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:11,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:11,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:11,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:11,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:11,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:11,981 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:11,981 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:11,981 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:11,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:11,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:11,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:11,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:11,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:11,981 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 14:54:11,982 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 14:54:11,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:11,982 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 14:54:11,982 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 14:54:11,982 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 14:54:11,982 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 14:54:11,982 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 14:54:11,982 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:11,982 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 14:54:11,983 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 14:54:11,983 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 14:54:11,982 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 14:54:11,983 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 14:54:11,983 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 14:54:11,983 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 14:54:11,983 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:11,983 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:11,984 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 14:54:11,983 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 14:54:11,984 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 14:54:11,984 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 14:54:11,984 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 14:54:11,984 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:11,985 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:11,985 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. 2023-02-09 14:54:11,985 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. started... 2023-02-09 14:54:11,985 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2f41522f4f390d8b073e3a5237c42fff 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:54:11,986 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:11,986 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:11,986 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:11,986 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:11,987 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:11,987 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:11,996 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf1/35c28684129d4100bc06837375c9fcfc 2023-02-09 14:54:12,011 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf2/71f26af779b74c70842bff5a96f82839 2023-02-09 14:54:12,016 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf1/35c28684129d4100bc06837375c9fcfc as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/35c28684129d4100bc06837375c9fcfc 2023-02-09 14:54:12,021 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/35c28684129d4100bc06837375c9fcfc, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 14:54:12,022 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf2/71f26af779b74c70842bff5a96f82839 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/71f26af779b74c70842bff5a96f82839 2023-02-09 14:54:12,027 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/71f26af779b74c70842bff5a96f82839, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 14:54:12,028 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2f41522f4f390d8b073e3a5237c42fff in 43ms, sequenceid=19, compaction requested=false 2023-02-09 14:54:12,028 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2f41522f4f390d8b073e3a5237c42fff: 2023-02-09 14:54:12,028 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. 2023-02-09 14:54:12,028 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:12,028 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:12,028 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,028 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 14:54:12,028 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 14:54:12,036 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,036 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:12,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:12,036 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,036 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:12,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:12,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:12,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:12,038 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-09 14:54:12,038 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4900036d[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:12,038 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-09 14:54:12,038 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,039 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,039 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,039 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,039 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,039 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 14:54:12,039 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 14:54:12,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,040 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:12,039 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:12,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,040 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 14:54:12,040 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 14:54:12,040 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 14:54:12,040 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:12,040 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 14:54:12,040 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:12,041 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 14:54:12,041 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,041 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:12,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:12,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:12,041 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 14:54:12,046 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 14:54:12,046 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:12,046 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 14:54:12,047 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 14:54:12,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:12,047 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:12,047 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 14:54:12,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:12,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:12,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,41077,1675954443212': 2023-02-09 14:54:12,050 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-09 14:54:12,050 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-09 14:54:12,050 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:12,050 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:12,050 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-09 14:54:12,050 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:12,052 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,052 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,052 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,052 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,052 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,052 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:12,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:12,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,052 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,052 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:12,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,053 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:12,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,054 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:12,055 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,056 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,056 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,058 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,058 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,058 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,059 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:12,058 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,058 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,059 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,059 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-09 14:54:12,059 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:12,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,059 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,059 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,059 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,060 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:12,060 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:12,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,060 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,060 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,060 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,060 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,061 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,061 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,061 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,061 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,144 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-09 14:54:12,145 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-09 14:54:12,146 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:54:12,146 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-02-09 14:54:12,160 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:12,160 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:12,160 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:12,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:12,178 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:12,178 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:12,179 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:12,181 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:12,181 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-09 14:54:12,181 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-09 14:54:12,181 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:12,181 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-09 14:54:12,182 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-09 14:54:12,182 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,182 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,184 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,184 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,184 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,184 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,185 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,185 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:12,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,185 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,185 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,185 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:12,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:12,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:12,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,186 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 14:54:12,186 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 14:54:12,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,186 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 14:54:12,186 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 14:54:12,186 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 14:54:12,186 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 14:54:12,186 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 14:54:12,186 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 14:54:12,186 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 14:54:12,186 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 14:54:12,187 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:12,186 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 14:54:12,187 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 14:54:12,187 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 14:54:12,187 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:12,186 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:12,188 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 14:54:12,188 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 14:54:12,188 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 14:54:12,187 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 14:54:12,188 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 14:54:12,188 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 14:54:12,188 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:12,189 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:12,189 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. 2023-02-09 14:54:12,189 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. started... 2023-02-09 14:54:12,189 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2f41522f4f390d8b073e3a5237c42fff 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:54:12,190 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,190 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,190 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,190 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,190 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:12,190 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:12,205 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf1/982a4af6ef0e48139d05632e186d07ce 2023-02-09 14:54:12,224 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf2/ace0880e72924e5c88c7cdf409a5ceb0 2023-02-09 14:54:12,230 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf1/982a4af6ef0e48139d05632e186d07ce as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/982a4af6ef0e48139d05632e186d07ce 2023-02-09 14:54:12,236 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/982a4af6ef0e48139d05632e186d07ce, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 14:54:12,237 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf2/ace0880e72924e5c88c7cdf409a5ceb0 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ace0880e72924e5c88c7cdf409a5ceb0 2023-02-09 14:54:12,242 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ace0880e72924e5c88c7cdf409a5ceb0, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 14:54:12,243 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2f41522f4f390d8b073e3a5237c42fff in 54ms, sequenceid=32, compaction requested=true 2023-02-09 14:54:12,243 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2f41522f4f390d8b073e3a5237c42fff: 2023-02-09 14:54:12,243 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. 2023-02-09 14:54:12,243 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:12,243 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:12,243 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,243 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 14:54:12,243 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 14:54:12,245 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,245 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:12,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:12,245 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,245 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:12,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:12,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:12,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:12,247 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-09 14:54:12,247 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@31ddf1c3[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:12,247 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-09 14:54:12,247 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,249 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,249 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,249 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,249 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 14:54:12,249 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 14:54:12,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,249 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,249 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:12,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,249 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 14:54:12,249 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 14:54:12,249 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 14:54:12,249 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:12,249 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:12,249 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 14:54:12,249 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:12,252 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,252 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 14:54:12,252 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:12,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:12,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:12,252 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 14:54:12,253 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 14:54:12,253 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 14:54:12,253 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:12,253 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:12,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:12,253 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 14:54:12,254 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 14:54:12,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:12,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:12,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,41077,1675954443212': 2023-02-09 14:54:12,258 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-09 14:54:12,258 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-09 14:54:12,258 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:12,258 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:12,258 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-09 14:54:12,258 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:12,260 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,260 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,260 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,260 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,260 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:12,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:12,260 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,261 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,261 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:12,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,261 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,262 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:12,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:12,263 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,264 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,264 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,267 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,267 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,267 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,267 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,267 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,267 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,267 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,267 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,267 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,267 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:12,267 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,267 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,267 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,267 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,267 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,267 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-09 14:54:12,268 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:12,267 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,268 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,268 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,268 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,268 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,268 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,269 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:12,269 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:12,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,369 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:12,369 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:12,369 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:12,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:12,393 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:12,393 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:12,394 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:12,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:12,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-09 14:54:12,396 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-09 14:54:12,396 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:12,397 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-09 14:54:12,397 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-09 14:54:12,398 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,398 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,400 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,400 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,400 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,400 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,400 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,400 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:12,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,401 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,401 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,401 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:12,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,401 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 14:54:12,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:12,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:12,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,401 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 14:54:12,401 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 14:54:12,402 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 14:54:12,401 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 14:54:12,404 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 14:54:12,404 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:12,404 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 14:54:12,404 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 14:54:12,404 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 14:54:12,405 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 14:54:12,405 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 14:54:12,404 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 14:54:12,405 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 14:54:12,405 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 14:54:12,405 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:12,405 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 14:54:12,405 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 14:54:12,405 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:12,407 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:12,409 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. 2023-02-09 14:54:12,409 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 14:54:12,409 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. started... 2023-02-09 14:54:12,409 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:12,409 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 14:54:12,409 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 14:54:12,409 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2f41522f4f390d8b073e3a5237c42fff 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:54:12,410 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,411 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,411 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,411 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:12,411 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,411 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:12,422 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf1/340bfaba29544fd9bf87894b9a99da38 2023-02-09 14:54:12,442 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab 2023-02-09 14:54:12,447 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf1/340bfaba29544fd9bf87894b9a99da38 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/340bfaba29544fd9bf87894b9a99da38 2023-02-09 14:54:12,452 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/340bfaba29544fd9bf87894b9a99da38, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 14:54:12,453 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab 2023-02-09 14:54:12,458 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 14:54:12,459 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2f41522f4f390d8b073e3a5237c42fff in 50ms, sequenceid=45, compaction requested=true 2023-02-09 14:54:12,459 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2f41522f4f390d8b073e3a5237c42fff: 2023-02-09 14:54:12,459 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. 2023-02-09 14:54:12,459 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:12,459 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:12,459 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,459 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 14:54:12,459 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 14:54:12,462 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,462 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:12,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:12,462 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,462 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:12,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:12,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:12,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:12,464 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-09 14:54:12,464 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3ed0d08c[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:12,464 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-09 14:54:12,464 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,466 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,466 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,466 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,466 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 14:54:12,466 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 14:54:12,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,466 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,466 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 14:54:12,466 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 14:54:12,466 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:12,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,466 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 14:54:12,466 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 14:54:12,466 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:12,466 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:12,466 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:12,468 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 14:54:12,468 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:12,468 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 14:54:12,468 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 14:54:12,468 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:12,469 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 14:54:12,470 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,470 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 14:54:12,470 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:12,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:12,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:12,470 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 14:54:12,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:12,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:12,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:12,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,41077,1675954443212': 2023-02-09 14:54:12,475 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-09 14:54:12,475 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-09 14:54:12,475 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:12,475 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:12,475 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-09 14:54:12,475 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:12,478 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,478 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:12,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:12,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,478 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,478 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,478 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,478 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,478 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:12,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,478 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,479 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,479 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:12,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,481 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:12,481 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,482 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,485 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,485 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,485 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,485 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,485 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:12,485 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,485 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,485 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,485 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-09 14:54:12,486 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:12,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,485 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,485 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,486 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,486 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,486 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,486 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,486 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:12,486 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,486 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:12,486 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,487 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,487 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,586 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:12,587 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:12,587 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:12,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:12,607 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:12,607 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:12,608 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:12,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:12,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-09 14:54:12,611 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-09 14:54:12,611 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:12,611 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-09 14:54:12,611 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-09 14:54:12,611 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,611 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,612 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,612 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,612 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,612 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,613 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,613 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:12,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,613 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,614 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,614 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:12,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,614 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 14:54:12,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:12,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,614 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 14:54:12,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:12,614 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 14:54:12,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,614 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 14:54:12,614 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 14:54:12,615 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 14:54:12,615 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 14:54:12,615 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 14:54:12,615 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:12,614 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 14:54:12,615 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 14:54:12,615 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:12,615 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 14:54:12,615 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 14:54:12,615 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 14:54:12,615 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:12,615 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:12,615 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 14:54:12,615 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. 2023-02-09 14:54:12,615 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 14:54:12,615 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. started... 2023-02-09 14:54:12,615 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 14:54:12,615 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:12,615 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 14:54:12,615 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 14:54:12,615 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 14:54:12,615 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2f41522f4f390d8b073e3a5237c42fff 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:54:12,617 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,619 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,619 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,619 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:12,619 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,620 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:12,633 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf1/784f58a7a9e84904ba6ee620343e6abc 2023-02-09 14:54:12,651 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106 2023-02-09 14:54:12,657 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf1/784f58a7a9e84904ba6ee620343e6abc as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/784f58a7a9e84904ba6ee620343e6abc 2023-02-09 14:54:12,663 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/784f58a7a9e84904ba6ee620343e6abc, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 14:54:12,663 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106 2023-02-09 14:54:12,669 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 14:54:12,670 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2f41522f4f390d8b073e3a5237c42fff in 55ms, sequenceid=58, compaction requested=true 2023-02-09 14:54:12,670 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2f41522f4f390d8b073e3a5237c42fff: 2023-02-09 14:54:12,670 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. 2023-02-09 14:54:12,670 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:12,670 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:12,670 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,670 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 14:54:12,670 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 14:54:12,672 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,672 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:12,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:12,672 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,672 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:12,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:12,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:12,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:12,675 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-09 14:54:12,675 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6c93c5b0[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:12,675 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-09 14:54:12,675 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,676 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,676 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,676 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,676 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,676 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 14:54:12,676 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 14:54:12,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,676 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:12,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,676 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:12,677 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 14:54:12,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,677 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 14:54:12,677 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 14:54:12,677 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 14:54:12,677 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:12,677 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:12,678 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 14:54:12,678 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:12,678 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 14:54:12,680 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,680 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 14:54:12,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:12,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:12,680 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:12,680 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 14:54:12,680 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 14:54:12,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:12,680 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:12,680 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 14:54:12,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:12,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:12,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,41077,1675954443212': 2023-02-09 14:54:12,683 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-09 14:54:12,683 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-09 14:54:12,683 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:12,683 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:12,683 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-09 14:54:12,683 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:12,685 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,685 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,685 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,685 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,685 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,685 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,685 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,686 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:12,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:12,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,686 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:12,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,686 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:12,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,688 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:12,688 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,689 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,696 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,696 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,696 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,696 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,696 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,696 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,696 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,696 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,696 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,696 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,697 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-09 14:54:12,697 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:12,697 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,696 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:12,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,697 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,697 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,697 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,697 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:12,697 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:12,697 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,698 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,698 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,698 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,798 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:12,798 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:12,798 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:12,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:12,817 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:12,817 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:12,818 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:12,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:12,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-09 14:54:12,820 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-09 14:54:12,820 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:12,820 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-09 14:54:12,820 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-09 14:54:12,821 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,821 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,822 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,822 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,822 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,822 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,822 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,822 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:12,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,823 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,823 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,823 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:12,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:12,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:12,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,824 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 14:54:12,824 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 14:54:12,824 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-09 14:54:12,824 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 14:54:12,824 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 14:54:12,824 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 14:54:12,824 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 14:54:12,824 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 14:54:12,824 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 14:54:12,824 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:12,824 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-09 14:54:12,824 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 14:54:12,825 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 14:54:12,825 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 14:54:12,824 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:12,824 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-09 14:54:12,825 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 14:54:12,825 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 14:54:12,825 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 14:54:12,825 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-09 14:54:12,825 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:12,825 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-09 14:54:12,825 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:12,825 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:12,825 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. 2023-02-09 14:54:12,825 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. started... 2023-02-09 14:54:12,825 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2f41522f4f390d8b073e3a5237c42fff 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:54:12,826 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,827 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,827 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,827 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:12,827 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,827 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:12,840 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e 2023-02-09 14:54:12,855 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0 2023-02-09 14:54:12,860 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e 2023-02-09 14:54:12,866 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 14:54:12,867 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0 2023-02-09 14:54:12,872 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 14:54:12,873 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2f41522f4f390d8b073e3a5237c42fff in 48ms, sequenceid=71, compaction requested=true 2023-02-09 14:54:12,873 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2f41522f4f390d8b073e3a5237c42fff: 2023-02-09 14:54:12,873 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. 2023-02-09 14:54:12,874 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:12,874 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:12,874 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,874 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-09 14:54:12,874 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-09 14:54:12,876 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,876 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:12,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:12,877 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,877 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:12,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:12,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:12,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:12,878 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-09 14:54:12,878 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@25c1a824[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:12,879 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-09 14:54:12,879 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,880 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,880 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,880 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,880 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 14:54:12,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,880 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,880 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:12,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,880 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 14:54:12,880 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 14:54:12,880 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 14:54:12,880 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:12,880 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:12,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,880 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-09 14:54:12,880 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-09 14:54:12,880 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:12,881 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,881 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 14:54:12,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:12,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:12,882 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:12,881 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 14:54:12,882 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 14:54:12,882 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:12,882 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 14:54:12,882 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-09 14:54:12,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:12,882 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:12,882 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-09 14:54:12,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:12,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:12,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,41077,1675954443212': 2023-02-09 14:54:12,885 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-09 14:54:12,885 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-09 14:54:12,885 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:12,885 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:12,886 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-09 14:54:12,886 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:12,889 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,889 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,889 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:12,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:12,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,889 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,889 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,889 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:12,889 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,889 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,890 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:12,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,890 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:12,892 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-09 14:54:12,892 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,892 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,895 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,895 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,895 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,895 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,895 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,895 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,895 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:12,895 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-09 14:54:12,895 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,895 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-09 14:54:12,896 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:12,895 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:12,895 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,895 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:12,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,895 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:12,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:12,896 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,896 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:12,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,896 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:12,896 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:12,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:12,896 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:12,897 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-09 14:54:12,897 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-09 14:54:12,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:12,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:12,996 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:12,997 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:12,997 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:12,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:13,260 WARN [Listener at localhost/40823] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-09 14:54:13,261 DEBUG [Listener at localhost/40823] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:13,263 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:13,265 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:13,265 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:13,265 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:13,268 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:13,268 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:13,268 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:13,268 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:13,514 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:13,515 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:13,515 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:13,516 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:13,518 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-02-09 14:54:13,519 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:13,519 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:13,519 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:13,520 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:13,520 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:13,524 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-09 14:54:13,526 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:13,527 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:13,527 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:13,528 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:13,528 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:13,529 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:13,529 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:13,530 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:13,531 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:13,531 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:13,531 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:13,532 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:13,532 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:13,532 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:13,532 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-09 14:54:13,534 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:13,534 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:13,536 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954483536 2023-02-09 14:54:13,537 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:13,537 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:13,544 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:13,545 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=2f41522f4f390d8b073e3a5237c42fff, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:13,545 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:13,545 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954513545 2023-02-09 14:54:13,545 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954573545 2023-02-09 14:54:13,545 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 7 msec 2023-02-09 14:54:13,545 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:54:13,553 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:13,553 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:13,553 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:13,553 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:13,795 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:13,795 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:13,795 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:13,796 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:13,796 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:54:13,796 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:13,796 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:13,796 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:13,796 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:13,797 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:13,798 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-09 14:54:13,799 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:13,799 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:13,800 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:13,800 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:13,800 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:13,800 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:13,801 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:13,801 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:13,801 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:13,802 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:13,802 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:13,802 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:13,802 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:13,802 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:13,802 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 14:54:13,803 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:13,803 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:13,803 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954483803 2023-02-09 14:54:13,803 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:13,805 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:13,807 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:13,808 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=2f41522f4f390d8b073e3a5237c42fff, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:13,808 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:13,808 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954513808 2023-02-09 14:54:13,808 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954573808 2023-02-09 14:54:13,808 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 14:54:13,808 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:54:13,811 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN}] 2023-02-09 14:54:13,811 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:13,811 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:13,811 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:13,811 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:14,058 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:14,059 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:14,059 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:14,059 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:14,059 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:54:14,059 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:14,059 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:14,059 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:14,060 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:14,060 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:14,062 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-09 14:54:14,062 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:14,063 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:14,063 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:14,063 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:14,064 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:14,064 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:14,064 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:14,064 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:14,065 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:14,065 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:14,065 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:14,065 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:14,065 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:14,065 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:14,065 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 14:54:14,066 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:14,067 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:14,067 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954484067 2023-02-09 14:54:14,067 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:14,069 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:14,071 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:14,071 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=2f41522f4f390d8b073e3a5237c42fff, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:14,072 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:14,072 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954514072 2023-02-09 14:54:14,072 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954574072 2023-02-09 14:54:14,072 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 14:54:14,072 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN}] 2023-02-09 14:54:14,072 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:54:14,074 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN 2023-02-09 14:54:14,075 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=2f41522f4f390d8b073e3a5237c42fff, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:14,075 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675954454075"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954454075"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954454075"}]},"ts":"1675954454075"} 2023-02-09 14:54:14,077 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 2f41522f4f390d8b073e3a5237c42fff, server=jenkins-hbase4.apache.org,41077,1675954443212}] 2023-02-09 14:54:14,077 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:14,077 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN 2023-02-09 14:54:14,077 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:14,077 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:14,322 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:14,322 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:14,322 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:14,322 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:14,324 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:54:14,324 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:14,324 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:14,324 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:14,324 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:14,325 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:14,326 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 14:54:14,326 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-09 14:54:14,327 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:14,327 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:14,327 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:14,328 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:14,328 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:14,329 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:14,329 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:14,329 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:14,329 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:14,330 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:14,331 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN 2023-02-09 14:54:14,331 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:14,331 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:14,331 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:14,331 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:14,331 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:14,331 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN 2023-02-09 14:54:14,331 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 14:54:14,332 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:14,333 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:14,333 INFO [Listener at localhost/40823] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff to restore RIT 2023-02-09 14:54:14,333 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954484333 2023-02-09 14:54:14,333 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:14,334 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN 2023-02-09 14:54:14,337 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:14,337 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=2f41522f4f390d8b073e3a5237c42fff, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:14,337 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:14,337 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954514337 2023-02-09 14:54:14,337 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954574337 2023-02-09 14:54:14,338 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 14:54:14,338 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=2f41522f4f390d8b073e3a5237c42fff, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:14,338 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675954454338"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954454338"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954454338"}]},"ts":"1675954454338"} 2023-02-09 14:54:14,340 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 2f41522f4f390d8b073e3a5237c42fff, server=jenkins-hbase4.apache.org,41077,1675954443212}] 2023-02-09 14:54:14,340 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:54:14,495 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:14,495 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 2f41522f4f390d8b073e3a5237c42fff, disabling compactions & flushes 2023-02-09 14:54:14,495 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. 2023-02-09 14:54:14,495 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. 2023-02-09 14:54:14,495 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. after waiting 0 ms 2023-02-09 14:54:14,495 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. 2023-02-09 14:54:14,495 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 2f41522f4f390d8b073e3a5237c42fff 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 14:54:14,507 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf1/2b649a280ad64e18817ae22baabaf44a 2023-02-09 14:54:14,523 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf2/d7550d6f781e465f9c9cbaa270fd52a2 2023-02-09 14:54:14,529 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf1/2b649a280ad64e18817ae22baabaf44a as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/2b649a280ad64e18817ae22baabaf44a 2023-02-09 14:54:14,533 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/2b649a280ad64e18817ae22baabaf44a, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 14:54:14,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/.tmp/cf2/d7550d6f781e465f9c9cbaa270fd52a2 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/d7550d6f781e465f9c9cbaa270fd52a2 2023-02-09 14:54:14,538 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/d7550d6f781e465f9c9cbaa270fd52a2, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 14:54:14,539 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 2f41522f4f390d8b073e3a5237c42fff in 44ms, sequenceid=82, compaction requested=true 2023-02-09 14:54:14,560 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 14:54:14,561 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:54:14,561 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. 2023-02-09 14:54:14,561 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 2f41522f4f390d8b073e3a5237c42fff: Waiting for close lock at 1675954454495Running coprocessor pre-close hooks at 1675954454495Disabling compacts and flushes for region at 1675954454495Disabling writes for close at 1675954454495Obtaining lock to block concurrent updates at 1675954454495Preparing flush snapshotting stores in 2f41522f4f390d8b073e3a5237c42fff at 1675954454495Finished memstore snapshotting testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675954454496 (+1 ms)Flushing stores of testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. at 1675954454496Flushing 2f41522f4f390d8b073e3a5237c42fff/cf1: creating writer at 1675954454497 (+1 ms)Flushing 2f41522f4f390d8b073e3a5237c42fff/cf1: appending metadata at 1675954454499 (+2 ms)Flushing 2f41522f4f390d8b073e3a5237c42fff/cf1: closing flushed file at 1675954454499Flushing 2f41522f4f390d8b073e3a5237c42fff/cf2: creating writer at 1675954454513 (+14 ms)Flushing 2f41522f4f390d8b073e3a5237c42fff/cf2: appending metadata at 1675954454515 (+2 ms)Flushing 2f41522f4f390d8b073e3a5237c42fff/cf2: closing flushed file at 1675954454515Flushing 2f41522f4f390d8b073e3a5237c42fff/cf1: reopening flushed file at 1675954454530 (+15 ms)Flushing 2f41522f4f390d8b073e3a5237c42fff/cf2: reopening flushed file at 1675954454535 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 2f41522f4f390d8b073e3a5237c42fff in 44ms, sequenceid=82, compaction requested=true at 1675954454540 (+5 ms)Writing region close event to WAL at 1675954454557 (+17 ms)Running coprocessor post-close hooks at 1675954454561 (+4 ms)Closed at 1675954454561 2023-02-09 14:54:14,564 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:14,564 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=2f41522f4f390d8b073e3a5237c42fff, regionState=CLOSED 2023-02-09 14:54:14,564 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675954454564"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954454564"}]},"ts":"1675954454564"} 2023-02-09 14:54:14,566 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:14,566 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 2f41522f4f390d8b073e3a5237c42fff, server=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:14,566 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:14,566 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 2f41522f4f390d8b073e3a5237c42fff, server=jenkins-hbase4.apache.org,41077,1675954443212 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:14,588 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:14,588 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:14,588 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:14,588 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:14,589 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:54:14,589 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:14,589 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:14,589 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:14,589 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:14,590 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:14,591 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 14:54:14,591 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-09 14:54:14,591 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-09 14:54:14,591 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:14,591 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:14,592 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:14,592 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:14,592 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:14,593 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:14,593 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:14,593 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:14,593 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:14,594 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:14,594 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN 2023-02-09 14:54:14,594 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 2f41522f4f390d8b073e3a5237c42fff, server=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:14,594 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:14,594 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:14,594 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:14,594 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:14,594 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:14,595 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN 2023-02-09 14:54:14,595 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 2f41522f4f390d8b073e3a5237c42fff, server=jenkins-hbase4.apache.org,41077,1675954443212 held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:14,595 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 14:54:14,595 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:14,596 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:14,596 INFO [Listener at localhost/40823] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff to restore RIT 2023-02-09 14:54:14,596 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954484596 2023-02-09 14:54:14,596 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:14,597 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 2f41522f4f390d8b073e3a5237c42fff, server=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:14,599 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:14,599 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=2f41522f4f390d8b073e3a5237c42fff, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:14,599 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:14,599 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954514599 2023-02-09 14:54:14,599 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954574599 2023-02-09 14:54:14,599 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 14:54:14,600 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=2f41522f4f390d8b073e3a5237c42fff, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:14,600 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675954454600"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954454600"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954454600"}]},"ts":"1675954454600"} 2023-02-09 14:54:14,601 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:54:14,604 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-02-09 14:54:14,604 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 2f41522f4f390d8b073e3a5237c42fff, server=jenkins-hbase4.apache.org,41077,1675954443212 in 261 msec 2023-02-09 14:54:14,604 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:14,604 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN 2023-02-09 14:54:14,604 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:14,604 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:14,850 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:14,850 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:14,850 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:14,850 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:14,851 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:54:14,851 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:14,851 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:14,851 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:14,851 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:14,852 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:14,853 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 14:54:14,853 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-09 14:54:14,853 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 14:54:14,854 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:14,854 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:14,855 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:14,855 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:14,855 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:14,855 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:14,856 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:14,856 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:14,856 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:14,857 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:14,857 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN 2023-02-09 14:54:14,857 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 2f41522f4f390d8b073e3a5237c42fff, server=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:14,857 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:14,857 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:14,857 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:14,857 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:14,857 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:14,857 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 2f41522f4f390d8b073e3a5237c42fff, server=jenkins-hbase4.apache.org,41077,1675954443212 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:14,857 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 14:54:14,858 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:14,859 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:14,859 INFO [Listener at localhost/40823] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff to restore RIT 2023-02-09 14:54:14,859 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954484859 2023-02-09 14:54:14,859 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:14,860 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN 2023-02-09 14:54:14,862 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:14,862 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=2f41522f4f390d8b073e3a5237c42fff, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:14,862 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:14,863 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954514863 2023-02-09 14:54:14,863 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954574863 2023-02-09 14:54:14,863 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 14:54:14,863 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN 2023-02-09 14:54:14,864 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:54:14,868 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-02-09 14:54:14,868 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN in 792 msec 2023-02-09 14:54:14,871 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:14,871 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:14,871 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:14,871 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:15,113 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:15,113 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:15,113 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:15,113 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:15,113 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:54:15,114 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:15,114 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:15,114 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:15,114 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:15,115 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:15,116 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 14:54:15,117 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 14:54:15,117 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 14:54:15,117 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:15,118 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:15,118 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:15,118 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:15,118 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:15,119 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:15,119 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:15,119 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:15,120 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:15,121 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:15,121 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN 2023-02-09 14:54:15,121 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 2f41522f4f390d8b073e3a5237c42fff, server=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:15,121 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:15,121 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:15,121 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 2f41522f4f390d8b073e3a5237c42fff, server=jenkins-hbase4.apache.org,41077,1675954443212 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:15,122 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 14:54:15,122 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:15,123 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:15,123 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954485123 2023-02-09 14:54:15,124 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:15,125 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:15,128 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:15,128 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=2f41522f4f390d8b073e3a5237c42fff, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:15,128 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:15,129 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954515128 2023-02-09 14:54:15,129 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954575129 2023-02-09 14:54:15,129 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 14:54:15,129 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:15,129 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:15,129 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:15,134 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:54:15,145 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=2f41522f4f390d8b073e3a5237c42fff, threads=14 2023-02-09 14:54:15,147 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,147 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/71f26af779b74c70842bff5a96f82839 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,147 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ace0880e72924e5c88c7cdf409a5ceb0 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,147 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,148 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,148 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/d7550d6f781e465f9c9cbaa270fd52a2 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,149 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,149 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/24d455dade92499595cdab11694c97e3 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,149 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,149 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/2b649a280ad64e18817ae22baabaf44a for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,149 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/340bfaba29544fd9bf87894b9a99da38 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,149 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/35c28684129d4100bc06837375c9fcfc for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,150 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/784f58a7a9e84904ba6ee620343e6abc for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,150 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/982a4af6ef0e48139d05632e186d07ce for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,264 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,285 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/982a4af6ef0e48139d05632e186d07ce for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,293 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,294 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ace0880e72924e5c88c7cdf409a5ceb0 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,295 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/d7550d6f781e465f9c9cbaa270fd52a2 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,309 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/2b649a280ad64e18817ae22baabaf44a for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,316 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/340bfaba29544fd9bf87894b9a99da38 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,316 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/24d455dade92499595cdab11694c97e3 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,317 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,320 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,323 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,324 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/784f58a7a9e84904ba6ee620343e6abc for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,325 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/71f26af779b74c70842bff5a96f82839 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,325 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/35c28684129d4100bc06837375c9fcfc for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,325 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 2f41522f4f390d8b073e3a5237c42fff Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 14:54:15,363 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:15,363 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:15,363 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:15,363 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:15,379 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:15,379 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:15,379 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:15,379 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:15,380 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:54:15,380 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:15,380 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:15,381 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:15,381 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:15,381 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:15,382 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-09 14:54:15,382 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 14:54:15,382 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 14:54:15,383 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:15,383 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:15,383 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:15,384 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:15,384 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:15,385 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:15,385 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:15,385 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:15,386 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:15,386 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:15,387 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN 2023-02-09 14:54:15,387 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 2f41522f4f390d8b073e3a5237c42fff, server=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:15,387 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:15,387 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:15,387 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:15,387 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:15,387 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:15,387 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 2f41522f4f390d8b073e3a5237c42fff, server=jenkins-hbase4.apache.org,41077,1675954443212 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:15,387 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 14:54:15,388 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:15,388 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:15,389 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954485388 2023-02-09 14:54:15,389 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:15,390 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:15,392 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:15,392 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=2f41522f4f390d8b073e3a5237c42fff, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:15,392 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:15,392 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954515392 2023-02-09 14:54:15,392 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954575392 2023-02-09 14:54:15,393 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 14:54:15,403 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=2f41522f4f390d8b073e3a5237c42fff, threads=14 2023-02-09 14:54:15,404 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,404 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/71f26af779b74c70842bff5a96f82839 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,404 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ace0880e72924e5c88c7cdf409a5ceb0 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,404 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,405 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,408 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/d7550d6f781e465f9c9cbaa270fd52a2 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,408 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,408 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,408 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/24d455dade92499595cdab11694c97e3 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,409 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/2b649a280ad64e18817ae22baabaf44a for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,409 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/340bfaba29544fd9bf87894b9a99da38 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,409 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/35c28684129d4100bc06837375c9fcfc for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,412 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/784f58a7a9e84904ba6ee620343e6abc for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,416 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/982a4af6ef0e48139d05632e186d07ce for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,556 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ace0880e72924e5c88c7cdf409a5ceb0 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,557 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/35c28684129d4100bc06837375c9fcfc for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,576 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,576 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,596 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,596 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/340bfaba29544fd9bf87894b9a99da38 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,600 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/71f26af779b74c70842bff5a96f82839 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,628 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,628 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/982a4af6ef0e48139d05632e186d07ce for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,634 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,635 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/784f58a7a9e84904ba6ee620343e6abc for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,638 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/d7550d6f781e465f9c9cbaa270fd52a2 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,639 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/24d455dade92499595cdab11694c97e3 for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,641 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/2b649a280ad64e18817ae22baabaf44a for region: 2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:15,641 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 2f41522f4f390d8b073e3a5237c42fff Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 14:54:15,703 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:54:15,709 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 14:54:15,711 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 14:54:15,711 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:15,711 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:15,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:15,711 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:15,893 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:15,893 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:15,893 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:15,893 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:15,893 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:54:15,893 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:15,894 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:15,894 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:15,894 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:15,894 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:15,896 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-09 14:54:15,896 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 14:54:15,896 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 14:54:15,896 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:15,896 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:15,897 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:15,897 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:15,897 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:15,897 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:15,897 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:15,898 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:15,898 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:15,898 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:15,898 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN 2023-02-09 14:54:15,899 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 2f41522f4f390d8b073e3a5237c42fff, server=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:15,899 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:15,899 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:15,899 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:15,899 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:15,899 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:15,899 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 2f41522f4f390d8b073e3a5237c42fff, server=jenkins-hbase4.apache.org,41077,1675954443212 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:15,899 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 14:54:15,900 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:15,900 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:15,901 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954485901 2023-02-09 14:54:15,902 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:15,902 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:15,904 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:15,905 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=2f41522f4f390d8b073e3a5237c42fff, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:15,905 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:15,905 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954515905 2023-02-09 14:54:15,905 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954575905 2023-02-09 14:54:15,905 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 14:54:15,908 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:54:15,910 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:15,910 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:15,910 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:15,910 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:16,155 WARN [Listener at localhost/40823] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 14:54:16,155 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:16,155 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:16,155 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:16,156 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:54:16,156 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:16,156 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:16,156 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:16,156 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:16,157 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:16,158 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-09 14:54:16,158 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 14:54:16,159 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 14:54:16,159 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:16,160 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:16,160 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:16,160 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:16,161 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:16,161 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:16,161 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:16,162 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:16,162 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:16,163 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:16,163 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN 2023-02-09 14:54:16,163 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 2f41522f4f390d8b073e3a5237c42fff, server=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:16,163 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:16,163 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:16,163 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:16,164 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:16,164 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=2f41522f4f390d8b073e3a5237c42fff, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:16,164 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 2f41522f4f390d8b073e3a5237c42fff, server=jenkins-hbase4.apache.org,41077,1675954443212 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:16,164 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 14:54:16,165 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:16,165 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:16,166 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954486166 2023-02-09 14:54:16,167 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:16,167 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:16,170 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:16,170 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=2f41522f4f390d8b073e3a5237c42fff, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:16,170 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:16,170 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954516170 2023-02-09 14:54:16,170 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954576170 2023-02-09 14:54:16,170 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 14:54:16,173 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675954456173"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1675954456173"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1675954456173"}]},"ts":"1675954456173"} 2023-02-09 14:54:16,173 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675954456173"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954456173"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954456173"}]},"ts":"1675954456173"} 2023-02-09 14:54:16,173 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675954456173"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954456173"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954456173"}]},"ts":"1675954456173"} 2023-02-09 14:54:16,179 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=f9cd34e6dad51e106afa9855e7d82622, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=10ba4d60d6f9660fb4081f6b504506d4, ASSIGN}] 2023-02-09 14:54:16,181 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=10ba4d60d6f9660fb4081f6b504506d4, ASSIGN 2023-02-09 14:54:16,182 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=10ba4d60d6f9660fb4081f6b504506d4, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41077,1675954443212; forceNewPlan=false, retain=false 2023-02-09 14:54:16,183 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=f9cd34e6dad51e106afa9855e7d82622, ASSIGN 2023-02-09 14:54:16,184 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=f9cd34e6dad51e106afa9855e7d82622, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41077,1675954443212; forceNewPlan=false, retain=false 2023-02-09 14:54:16,320 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 14:54:16,332 INFO [jenkins-hbase4:43829] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 14:54:16,334 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=10ba4d60d6f9660fb4081f6b504506d4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:16,334 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675954456334"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954456334"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954456334"}]},"ts":"1675954456334"} 2023-02-09 14:54:16,366 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=32, state=RUNNABLE; OpenRegionProcedure 10ba4d60d6f9660fb4081f6b504506d4, server=jenkins-hbase4.apache.org,41077,1675954443212}] 2023-02-09 14:54:16,371 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=f9cd34e6dad51e106afa9855e7d82622, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:16,372 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675954456371"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954456371"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954456371"}]},"ts":"1675954456371"} 2023-02-09 14:54:16,380 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=31, state=RUNNABLE; OpenRegionProcedure f9cd34e6dad51e106afa9855e7d82622, server=jenkins-hbase4.apache.org,41077,1675954443212}] 2023-02-09 14:54:16,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622. 2023-02-09 14:54:16,523 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f9cd34e6dad51e106afa9855e7d82622, NAME => 'testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.', STARTKEY => '', ENDKEY => '41'} 2023-02-09 14:54:16,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:54:16,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR f9cd34e6dad51e106afa9855e7d82622 2023-02-09 14:54:16,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:16,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for f9cd34e6dad51e106afa9855e7d82622 2023-02-09 14:54:16,524 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for f9cd34e6dad51e106afa9855e7d82622 2023-02-09 14:54:16,525 INFO [StoreOpener-f9cd34e6dad51e106afa9855e7d82622-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f9cd34e6dad51e106afa9855e7d82622 2023-02-09 14:54:16,527 DEBUG [StoreOpener-f9cd34e6dad51e106afa9855e7d82622-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1 2023-02-09 14:54:16,527 DEBUG [StoreOpener-f9cd34e6dad51e106afa9855e7d82622-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1 2023-02-09 14:54:16,527 INFO [StoreOpener-f9cd34e6dad51e106afa9855e7d82622-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f9cd34e6dad51e106afa9855e7d82622 columnFamilyName cf1 2023-02-09 14:54:16,544 DEBUG [StoreOpener-f9cd34e6dad51e106afa9855e7d82622-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e-bottom 2023-02-09 14:54:16,549 DEBUG [StoreOpener-f9cd34e6dad51e106afa9855e7d82622-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/24d455dade92499595cdab11694c97e3.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/24d455dade92499595cdab11694c97e3-bottom 2023-02-09 14:54:16,553 DEBUG [StoreOpener-f9cd34e6dad51e106afa9855e7d82622-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/2b649a280ad64e18817ae22baabaf44a.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/2b649a280ad64e18817ae22baabaf44a-bottom 2023-02-09 14:54:16,559 DEBUG [StoreOpener-f9cd34e6dad51e106afa9855e7d82622-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/340bfaba29544fd9bf87894b9a99da38.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/340bfaba29544fd9bf87894b9a99da38-bottom 2023-02-09 14:54:16,565 DEBUG [StoreOpener-f9cd34e6dad51e106afa9855e7d82622-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/35c28684129d4100bc06837375c9fcfc.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/35c28684129d4100bc06837375c9fcfc-bottom 2023-02-09 14:54:16,570 DEBUG [StoreOpener-f9cd34e6dad51e106afa9855e7d82622-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/784f58a7a9e84904ba6ee620343e6abc.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/784f58a7a9e84904ba6ee620343e6abc-bottom 2023-02-09 14:54:16,574 DEBUG [StoreOpener-f9cd34e6dad51e106afa9855e7d82622-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/982a4af6ef0e48139d05632e186d07ce.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/982a4af6ef0e48139d05632e186d07ce-bottom 2023-02-09 14:54:16,575 INFO [StoreOpener-f9cd34e6dad51e106afa9855e7d82622-1] regionserver.HStore(310): Store=f9cd34e6dad51e106afa9855e7d82622/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:16,575 INFO [StoreOpener-f9cd34e6dad51e106afa9855e7d82622-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f9cd34e6dad51e106afa9855e7d82622 2023-02-09 14:54:16,576 DEBUG [StoreOpener-f9cd34e6dad51e106afa9855e7d82622-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2 2023-02-09 14:54:16,576 DEBUG [StoreOpener-f9cd34e6dad51e106afa9855e7d82622-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2 2023-02-09 14:54:16,576 INFO [StoreOpener-f9cd34e6dad51e106afa9855e7d82622-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f9cd34e6dad51e106afa9855e7d82622 columnFamilyName cf2 2023-02-09 14:54:16,591 DEBUG [StoreOpener-f9cd34e6dad51e106afa9855e7d82622-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0-bottom 2023-02-09 14:54:16,596 DEBUG [StoreOpener-f9cd34e6dad51e106afa9855e7d82622-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/71f26af779b74c70842bff5a96f82839.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/71f26af779b74c70842bff5a96f82839-bottom 2023-02-09 14:54:16,600 DEBUG [StoreOpener-f9cd34e6dad51e106afa9855e7d82622-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/ace0880e72924e5c88c7cdf409a5ceb0.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ace0880e72924e5c88c7cdf409a5ceb0-bottom 2023-02-09 14:54:16,606 DEBUG [StoreOpener-f9cd34e6dad51e106afa9855e7d82622-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d-bottom 2023-02-09 14:54:16,611 DEBUG [StoreOpener-f9cd34e6dad51e106afa9855e7d82622-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab-bottom 2023-02-09 14:54:16,616 DEBUG [StoreOpener-f9cd34e6dad51e106afa9855e7d82622-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/d7550d6f781e465f9c9cbaa270fd52a2.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/d7550d6f781e465f9c9cbaa270fd52a2-bottom 2023-02-09 14:54:16,621 DEBUG [StoreOpener-f9cd34e6dad51e106afa9855e7d82622-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106-bottom 2023-02-09 14:54:16,621 INFO [StoreOpener-f9cd34e6dad51e106afa9855e7d82622-1] regionserver.HStore(310): Store=f9cd34e6dad51e106afa9855e7d82622/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:16,622 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622 2023-02-09 14:54:16,623 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622 2023-02-09 14:54:16,625 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:54:16,627 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for f9cd34e6dad51e106afa9855e7d82622 2023-02-09 14:54:16,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened f9cd34e6dad51e106afa9855e7d82622; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74705868, jitterRate=0.11320418119430542}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:54:16,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for f9cd34e6dad51e106afa9855e7d82622: Running coprocessor pre-open hook at 1675954456524Writing region info on filesystem at 1675954456524Initializing all the Stores at 1675954456525 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954456525Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954456525Cleaning up temporary data from old regions at 1675954456624 (+99 ms)Cleaning up detritus from prior splits at 1675954456624Running coprocessor post-open hooks at 1675954456628 (+4 ms)Region opened successfully at 1675954456628 2023-02-09 14:54:16,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622., pid=34, masterSystemTime=1675954456520 2023-02-09 14:54:16,629 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:16,629 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 14:54:16,630 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 14:54:16,630 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 14:54:16,630 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622. 2023-02-09 14:54:16,630 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622. 2023-02-09 14:54:16,630 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStore(1912): f9cd34e6dad51e106afa9855e7d82622/cf1 is initiating minor compaction (all files) 2023-02-09 14:54:16,630 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStore(1912): f9cd34e6dad51e106afa9855e7d82622/cf2 is initiating minor compaction (all files) 2023-02-09 14:54:16,630 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of f9cd34e6dad51e106afa9855e7d82622/cf1 in testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622. 2023-02-09 14:54:16,630 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HRegion(2258): Starting compaction of f9cd34e6dad51e106afa9855e7d82622/cf2 in testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622. 2023-02-09 14:54:16,631 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/24d455dade92499595cdab11694c97e3.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/24d455dade92499595cdab11694c97e3-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/35c28684129d4100bc06837375c9fcfc.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/35c28684129d4100bc06837375c9fcfc-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/982a4af6ef0e48139d05632e186d07ce.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/982a4af6ef0e48139d05632e186d07ce-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/340bfaba29544fd9bf87894b9a99da38.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/340bfaba29544fd9bf87894b9a99da38-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/784f58a7a9e84904ba6ee620343e6abc.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/784f58a7a9e84904ba6ee620343e6abc-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/2b649a280ad64e18817ae22baabaf44a.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/2b649a280ad64e18817ae22baabaf44a-bottom] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/.tmp, totalSize=34.9 K 2023-02-09 14:54:16,631 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/71f26af779b74c70842bff5a96f82839.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/71f26af779b74c70842bff5a96f82839-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/ace0880e72924e5c88c7cdf409a5ceb0.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ace0880e72924e5c88c7cdf409a5ceb0-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/d7550d6f781e465f9c9cbaa270fd52a2.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/d7550d6f781e465f9c9cbaa270fd52a2-bottom] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/.tmp, totalSize=34.9 K 2023-02-09 14:54:16,631 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 24d455dade92499595cdab11694c97e3.2f41522f4f390d8b073e3a5237c42fff, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675954451767 2023-02-09 14:54:16,632 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting b03ae9b135764b5f9c6b0a3c5f96de5d.2f41522f4f390d8b073e3a5237c42fff, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675954451767 2023-02-09 14:54:16,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622. 2023-02-09 14:54:16,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622. 2023-02-09 14:54:16,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4. 2023-02-09 14:54:16,632 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 35c28684129d4100bc06837375c9fcfc.2f41522f4f390d8b073e3a5237c42fff, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675954451958 2023-02-09 14:54:16,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 10ba4d60d6f9660fb4081f6b504506d4, NAME => 'testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.', STARTKEY => '41', ENDKEY => ''} 2023-02-09 14:54:16,633 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting 71f26af779b74c70842bff5a96f82839.2f41522f4f390d8b073e3a5237c42fff, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675954451958 2023-02-09 14:54:16,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:54:16,633 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 982a4af6ef0e48139d05632e186d07ce.2f41522f4f390d8b073e3a5237c42fff, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675954452162 2023-02-09 14:54:16,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:16,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:16,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:16,633 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting ace0880e72924e5c88c7cdf409a5ceb0.2f41522f4f390d8b073e3a5237c42fff, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675954452162 2023-02-09 14:54:16,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:16,633 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 340bfaba29544fd9bf87894b9a99da38.2f41522f4f390d8b073e3a5237c42fff, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675954452372 2023-02-09 14:54:16,634 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=f9cd34e6dad51e106afa9855e7d82622, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:16,634 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675954456634"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954456634"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954456634"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954456634"}]},"ts":"1675954456634"} 2023-02-09 14:54:16,634 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting ca9642f1f44d4e39b488e2e8fa97f0ab.2f41522f4f390d8b073e3a5237c42fff, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675954452372 2023-02-09 14:54:16,635 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting f9de83d1b2674c6bbda0f8d1b1ec7106.2f41522f4f390d8b073e3a5237c42fff, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675954452589 2023-02-09 14:54:16,636 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 784f58a7a9e84904ba6ee620343e6abc.2f41522f4f390d8b073e3a5237c42fff, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675954452589 2023-02-09 14:54:16,636 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting 46fbacd2dcbc4ea1b04e7a1b227349b0.2f41522f4f390d8b073e3a5237c42fff, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675954452800 2023-02-09 14:54:16,637 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 0be6e1aa2b3f4c87ab1ae4ba21138f9e.2f41522f4f390d8b073e3a5237c42fff, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675954452800 2023-02-09 14:54:16,637 INFO [StoreOpener-10ba4d60d6f9660fb4081f6b504506d4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:16,637 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting d7550d6f781e465f9c9cbaa270fd52a2.2f41522f4f390d8b073e3a5237c42fff, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675954452999 2023-02-09 14:54:16,638 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 2b649a280ad64e18817ae22baabaf44a.2f41522f4f390d8b073e3a5237c42fff, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675954452999 2023-02-09 14:54:16,639 DEBUG [StoreOpener-10ba4d60d6f9660fb4081f6b504506d4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1 2023-02-09 14:54:16,639 DEBUG [StoreOpener-10ba4d60d6f9660fb4081f6b504506d4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1 2023-02-09 14:54:16,639 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=31 2023-02-09 14:54:16,640 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=31, state=SUCCESS; OpenRegionProcedure f9cd34e6dad51e106afa9855e7d82622, server=jenkins-hbase4.apache.org,41077,1675954443212 in 256 msec 2023-02-09 14:54:16,640 INFO [StoreOpener-10ba4d60d6f9660fb4081f6b504506d4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 10ba4d60d6f9660fb4081f6b504506d4 columnFamilyName cf1 2023-02-09 14:54:16,642 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=f9cd34e6dad51e106afa9855e7d82622, ASSIGN in 463 msec 2023-02-09 14:54:16,659 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] throttle.PressureAwareThroughputController(145): f9cd34e6dad51e106afa9855e7d82622#cf2#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:54:16,664 DEBUG [StoreOpener-10ba4d60d6f9660fb4081f6b504506d4-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e-top 2023-02-09 14:54:16,668 DEBUG [StoreOpener-10ba4d60d6f9660fb4081f6b504506d4-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/24d455dade92499595cdab11694c97e3.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/24d455dade92499595cdab11694c97e3-top 2023-02-09 14:54:16,673 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] throttle.PressureAwareThroughputController(145): f9cd34e6dad51e106afa9855e7d82622#cf1#compaction#33 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:54:16,674 DEBUG [StoreOpener-10ba4d60d6f9660fb4081f6b504506d4-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/2b649a280ad64e18817ae22baabaf44a.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/2b649a280ad64e18817ae22baabaf44a-top 2023-02-09 14:54:16,723 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/.tmp/cf2/3530e87b21ad4cc8aa6e09c4338077f1 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/3530e87b21ad4cc8aa6e09c4338077f1 2023-02-09 14:54:16,726 DEBUG [StoreOpener-10ba4d60d6f9660fb4081f6b504506d4-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/340bfaba29544fd9bf87894b9a99da38.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/340bfaba29544fd9bf87894b9a99da38-top 2023-02-09 14:54:16,734 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f9cd34e6dad51e106afa9855e7d82622/cf2 of f9cd34e6dad51e106afa9855e7d82622 into 3530e87b21ad4cc8aa6e09c4338077f1(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:54:16,735 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for f9cd34e6dad51e106afa9855e7d82622: 2023-02-09 14:54:16,735 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622., storeName=f9cd34e6dad51e106afa9855e7d82622/cf2, priority=9, startTime=1675954456629; duration=0sec 2023-02-09 14:54:16,735 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:16,746 DEBUG [StoreOpener-10ba4d60d6f9660fb4081f6b504506d4-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/35c28684129d4100bc06837375c9fcfc.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/35c28684129d4100bc06837375c9fcfc-top 2023-02-09 14:54:16,753 DEBUG [StoreOpener-10ba4d60d6f9660fb4081f6b504506d4-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/784f58a7a9e84904ba6ee620343e6abc.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/784f58a7a9e84904ba6ee620343e6abc-top 2023-02-09 14:54:16,755 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/.tmp/cf1/75df77dbc1bd4662a24a9c233649296b as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/75df77dbc1bd4662a24a9c233649296b 2023-02-09 14:54:16,758 DEBUG [StoreOpener-10ba4d60d6f9660fb4081f6b504506d4-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/982a4af6ef0e48139d05632e186d07ce.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/982a4af6ef0e48139d05632e186d07ce-top 2023-02-09 14:54:16,758 INFO [StoreOpener-10ba4d60d6f9660fb4081f6b504506d4-1] regionserver.HStore(310): Store=10ba4d60d6f9660fb4081f6b504506d4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:16,759 INFO [StoreOpener-10ba4d60d6f9660fb4081f6b504506d4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:16,761 DEBUG [StoreOpener-10ba4d60d6f9660fb4081f6b504506d4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2 2023-02-09 14:54:16,761 DEBUG [StoreOpener-10ba4d60d6f9660fb4081f6b504506d4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2 2023-02-09 14:54:16,761 INFO [StoreOpener-10ba4d60d6f9660fb4081f6b504506d4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 10ba4d60d6f9660fb4081f6b504506d4 columnFamilyName cf2 2023-02-09 14:54:16,764 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f9cd34e6dad51e106afa9855e7d82622/cf1 of f9cd34e6dad51e106afa9855e7d82622 into 75df77dbc1bd4662a24a9c233649296b(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:54:16,764 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for f9cd34e6dad51e106afa9855e7d82622: 2023-02-09 14:54:16,764 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622., storeName=f9cd34e6dad51e106afa9855e7d82622/cf1, priority=9, startTime=1675954456629; duration=0sec 2023-02-09 14:54:16,764 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:16,782 DEBUG [StoreOpener-10ba4d60d6f9660fb4081f6b504506d4-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0-top 2023-02-09 14:54:16,786 DEBUG [StoreOpener-10ba4d60d6f9660fb4081f6b504506d4-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/71f26af779b74c70842bff5a96f82839.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/71f26af779b74c70842bff5a96f82839-top 2023-02-09 14:54:16,790 DEBUG [StoreOpener-10ba4d60d6f9660fb4081f6b504506d4-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/ace0880e72924e5c88c7cdf409a5ceb0.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ace0880e72924e5c88c7cdf409a5ceb0-top 2023-02-09 14:54:16,796 DEBUG [StoreOpener-10ba4d60d6f9660fb4081f6b504506d4-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d-top 2023-02-09 14:54:16,806 DEBUG [StoreOpener-10ba4d60d6f9660fb4081f6b504506d4-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab-top 2023-02-09 14:54:16,811 DEBUG [StoreOpener-10ba4d60d6f9660fb4081f6b504506d4-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/d7550d6f781e465f9c9cbaa270fd52a2.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/d7550d6f781e465f9c9cbaa270fd52a2-top 2023-02-09 14:54:16,816 DEBUG [StoreOpener-10ba4d60d6f9660fb4081f6b504506d4-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106-top 2023-02-09 14:54:16,817 INFO [StoreOpener-10ba4d60d6f9660fb4081f6b504506d4-1] regionserver.HStore(310): Store=10ba4d60d6f9660fb4081f6b504506d4/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:16,817 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:16,819 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:16,821 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:54:16,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:16,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 10ba4d60d6f9660fb4081f6b504506d4; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65949540, jitterRate=-0.01727527379989624}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:54:16,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 10ba4d60d6f9660fb4081f6b504506d4: Running coprocessor pre-open hook at 1675954456633Writing region info on filesystem at 1675954456633Initializing all the Stores at 1675954456636 (+3 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954456636Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954456636Cleaning up temporary data from old regions at 1675954456820 (+184 ms)Cleaning up detritus from prior splits at 1675954456820Running coprocessor post-open hooks at 1675954456823 (+3 ms)Region opened successfully at 1675954456823 2023-02-09 14:54:16,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4., pid=33, masterSystemTime=1675954456520 2023-02-09 14:54:16,824 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:16,825 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 14:54:16,825 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 14:54:16,825 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 14:54:16,825 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4. 2023-02-09 14:54:16,825 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStore(1912): 10ba4d60d6f9660fb4081f6b504506d4/cf1 is initiating minor compaction (all files) 2023-02-09 14:54:16,825 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4. 2023-02-09 14:54:16,825 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStore(1912): 10ba4d60d6f9660fb4081f6b504506d4/cf2 is initiating minor compaction (all files) 2023-02-09 14:54:16,825 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 10ba4d60d6f9660fb4081f6b504506d4/cf1 in testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4. 2023-02-09 14:54:16,826 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 10ba4d60d6f9660fb4081f6b504506d4/cf2 in testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4. 2023-02-09 14:54:16,826 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/24d455dade92499595cdab11694c97e3.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/24d455dade92499595cdab11694c97e3-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/35c28684129d4100bc06837375c9fcfc.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/35c28684129d4100bc06837375c9fcfc-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/982a4af6ef0e48139d05632e186d07ce.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/982a4af6ef0e48139d05632e186d07ce-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/340bfaba29544fd9bf87894b9a99da38.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/340bfaba29544fd9bf87894b9a99da38-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/784f58a7a9e84904ba6ee620343e6abc.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/784f58a7a9e84904ba6ee620343e6abc-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/2b649a280ad64e18817ae22baabaf44a.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/2b649a280ad64e18817ae22baabaf44a-top] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/.tmp, totalSize=34.9 K 2023-02-09 14:54:16,826 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/71f26af779b74c70842bff5a96f82839.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/71f26af779b74c70842bff5a96f82839-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/ace0880e72924e5c88c7cdf409a5ceb0.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ace0880e72924e5c88c7cdf409a5ceb0-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/d7550d6f781e465f9c9cbaa270fd52a2.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/d7550d6f781e465f9c9cbaa270fd52a2-top] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/.tmp, totalSize=34.9 K 2023-02-09 14:54:16,827 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 24d455dade92499595cdab11694c97e3.2f41522f4f390d8b073e3a5237c42fff, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675954451767 2023-02-09 14:54:16,827 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting b03ae9b135764b5f9c6b0a3c5f96de5d.2f41522f4f390d8b073e3a5237c42fff, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675954451767 2023-02-09 14:54:16,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4. 2023-02-09 14:54:16,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4. 2023-02-09 14:54:16,828 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 35c28684129d4100bc06837375c9fcfc.2f41522f4f390d8b073e3a5237c42fff, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675954451958 2023-02-09 14:54:16,828 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting 71f26af779b74c70842bff5a96f82839.2f41522f4f390d8b073e3a5237c42fff, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675954451958 2023-02-09 14:54:16,828 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=10ba4d60d6f9660fb4081f6b504506d4, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:16,828 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 982a4af6ef0e48139d05632e186d07ce.2f41522f4f390d8b073e3a5237c42fff, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675954452162 2023-02-09 14:54:16,828 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675954456828"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954456828"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954456828"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954456828"}]},"ts":"1675954456828"} 2023-02-09 14:54:16,828 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting ace0880e72924e5c88c7cdf409a5ceb0.2f41522f4f390d8b073e3a5237c42fff, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675954452162 2023-02-09 14:54:16,829 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 340bfaba29544fd9bf87894b9a99da38.2f41522f4f390d8b073e3a5237c42fff, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675954452372 2023-02-09 14:54:16,829 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting ca9642f1f44d4e39b488e2e8fa97f0ab.2f41522f4f390d8b073e3a5237c42fff, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675954452372 2023-02-09 14:54:16,829 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 784f58a7a9e84904ba6ee620343e6abc.2f41522f4f390d8b073e3a5237c42fff, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675954452589 2023-02-09 14:54:16,829 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting f9de83d1b2674c6bbda0f8d1b1ec7106.2f41522f4f390d8b073e3a5237c42fff, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675954452589 2023-02-09 14:54:16,830 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 0be6e1aa2b3f4c87ab1ae4ba21138f9e.2f41522f4f390d8b073e3a5237c42fff, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675954452800 2023-02-09 14:54:16,830 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting 46fbacd2dcbc4ea1b04e7a1b227349b0.2f41522f4f390d8b073e3a5237c42fff, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675954452800 2023-02-09 14:54:16,831 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 2b649a280ad64e18817ae22baabaf44a.2f41522f4f390d8b073e3a5237c42fff, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675954452999 2023-02-09 14:54:16,831 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting d7550d6f781e465f9c9cbaa270fd52a2.2f41522f4f390d8b073e3a5237c42fff, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675954452999 2023-02-09 14:54:16,835 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=32 2023-02-09 14:54:16,835 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=32, state=SUCCESS; OpenRegionProcedure 10ba4d60d6f9660fb4081f6b504506d4, server=jenkins-hbase4.apache.org,41077,1675954443212 in 464 msec 2023-02-09 14:54:16,838 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=28 2023-02-09 14:54:16,838 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=10ba4d60d6f9660fb4081f6b504506d4, ASSIGN in 658 msec 2023-02-09 14:54:16,839 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 in 3.5760 sec 2023-02-09 14:54:16,848 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] throttle.PressureAwareThroughputController(145): 10ba4d60d6f9660fb4081f6b504506d4#cf2#compaction#34 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:54:16,851 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] throttle.PressureAwareThroughputController(145): 10ba4d60d6f9660fb4081f6b504506d4#cf1#compaction#35 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:54:16,877 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/.tmp/cf2/41d13af2f65e41f5b42af331a3a4af4a as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/41d13af2f65e41f5b42af331a3a4af4a 2023-02-09 14:54:16,878 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/.tmp/cf1/4eb79868f33d4bf09a1498d56814c7cc as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/4eb79868f33d4bf09a1498d56814c7cc 2023-02-09 14:54:16,883 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 10ba4d60d6f9660fb4081f6b504506d4/cf2 of 10ba4d60d6f9660fb4081f6b504506d4 into 41d13af2f65e41f5b42af331a3a4af4a(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:54:16,883 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 10ba4d60d6f9660fb4081f6b504506d4: 2023-02-09 14:54:16,884 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4., storeName=10ba4d60d6f9660fb4081f6b504506d4/cf2, priority=9, startTime=1675954456825; duration=0sec 2023-02-09 14:54:16,884 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:16,885 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 10ba4d60d6f9660fb4081f6b504506d4/cf1 of 10ba4d60d6f9660fb4081f6b504506d4 into 4eb79868f33d4bf09a1498d56814c7cc(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:54:16,885 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 10ba4d60d6f9660fb4081f6b504506d4: 2023-02-09 14:54:16,885 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4., storeName=10ba4d60d6f9660fb4081f6b504506d4/cf1, priority=9, startTime=1675954456824; duration=0sec 2023-02-09 14:54:16,885 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:16,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41077] ipc.CallRunner(144): callId: 239 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:55216 deadline: 1675954516921, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff. is not online on jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:17,034 DEBUG [Listener at localhost/40823] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 14:54:17,041 INFO [Listener at localhost/40823] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-09 14:54:17,043 DEBUG [Listener at localhost/40823] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 14:54:17,061 WARN [Listener at localhost/40823] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 14:54:17,062 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:17,062 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:17,063 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:17,063 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:17,064 INFO [Listener at localhost/40823] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-02-09 14:54:17,064 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitWithoutPONR 2023-02-09 14:54:17,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:17,068 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:17,068 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:17,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 14:54:17,069 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954457069"}]},"ts":"1675954457069"} 2023-02-09 14:54:17,070 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-02-09 14:54:17,072 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-02-09 14:54:17,073 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=f9cd34e6dad51e106afa9855e7d82622, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=10ba4d60d6f9660fb4081f6b504506d4, UNASSIGN}] 2023-02-09 14:54:17,074 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=10ba4d60d6f9660fb4081f6b504506d4, UNASSIGN 2023-02-09 14:54:17,075 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=10ba4d60d6f9660fb4081f6b504506d4, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:17,075 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675954457075"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954457075"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954457075"}]},"ts":"1675954457075"} 2023-02-09 14:54:17,076 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure 10ba4d60d6f9660fb4081f6b504506d4, server=jenkins-hbase4.apache.org,41077,1675954443212}] 2023-02-09 14:54:17,078 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=f9cd34e6dad51e106afa9855e7d82622, UNASSIGN 2023-02-09 14:54:17,079 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=f9cd34e6dad51e106afa9855e7d82622, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:17,079 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675954457079"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954457079"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954457079"}]},"ts":"1675954457079"} 2023-02-09 14:54:17,081 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure f9cd34e6dad51e106afa9855e7d82622, server=jenkins-hbase4.apache.org,41077,1675954443212}] 2023-02-09 14:54:17,169 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:17,169 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:17,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 14:54:17,228 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:17,228 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 10ba4d60d6f9660fb4081f6b504506d4, disabling compactions & flushes 2023-02-09 14:54:17,228 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4. 2023-02-09 14:54:17,228 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4. 2023-02-09 14:54:17,228 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4. after waiting 0 ms 2023-02-09 14:54:17,228 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4. 2023-02-09 14:54:17,240 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/24d455dade92499595cdab11694c97e3.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/24d455dade92499595cdab11694c97e3-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/35c28684129d4100bc06837375c9fcfc.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/35c28684129d4100bc06837375c9fcfc-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/982a4af6ef0e48139d05632e186d07ce.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/982a4af6ef0e48139d05632e186d07ce-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/340bfaba29544fd9bf87894b9a99da38.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/340bfaba29544fd9bf87894b9a99da38-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/784f58a7a9e84904ba6ee620343e6abc.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/784f58a7a9e84904ba6ee620343e6abc-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/2b649a280ad64e18817ae22baabaf44a.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/2b649a280ad64e18817ae22baabaf44a-top] to archive 2023-02-09 14:54:17,241 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:17,243 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/24d455dade92499595cdab11694c97e3.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/24d455dade92499595cdab11694c97e3.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,244 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/35c28684129d4100bc06837375c9fcfc.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/35c28684129d4100bc06837375c9fcfc.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,246 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/982a4af6ef0e48139d05632e186d07ce.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/982a4af6ef0e48139d05632e186d07ce.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,247 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/340bfaba29544fd9bf87894b9a99da38.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/340bfaba29544fd9bf87894b9a99da38.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,248 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/784f58a7a9e84904ba6ee620343e6abc.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/784f58a7a9e84904ba6ee620343e6abc.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,249 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,250 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/2b649a280ad64e18817ae22baabaf44a.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/2b649a280ad64e18817ae22baabaf44a.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,252 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/71f26af779b74c70842bff5a96f82839.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/71f26af779b74c70842bff5a96f82839-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/ace0880e72924e5c88c7cdf409a5ceb0.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ace0880e72924e5c88c7cdf409a5ceb0-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/d7550d6f781e465f9c9cbaa270fd52a2.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/d7550d6f781e465f9c9cbaa270fd52a2-top] to archive 2023-02-09 14:54:17,253 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:17,255 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,256 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/71f26af779b74c70842bff5a96f82839.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/71f26af779b74c70842bff5a96f82839.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,258 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/ace0880e72924e5c88c7cdf409a5ceb0.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/ace0880e72924e5c88c7cdf409a5ceb0.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,259 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,260 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,261 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,262 DEBUG [StoreCloser-testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/d7550d6f781e465f9c9cbaa270fd52a2.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/d7550d6f781e465f9c9cbaa270fd52a2.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,266 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 14:54:17,266 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:54:17,267 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4. 2023-02-09 14:54:17,267 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 10ba4d60d6f9660fb4081f6b504506d4: Waiting for close lock at 1675954457228Running coprocessor pre-close hooks at 1675954457228Disabling compacts and flushes for region at 1675954457228Disabling writes for close at 1675954457228Writing region close event to WAL at 1675954457263 (+35 ms)Running coprocessor post-close hooks at 1675954457266 (+3 ms)Closed at 1675954457267 (+1 ms) 2023-02-09 14:54:17,268 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:17,268 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close f9cd34e6dad51e106afa9855e7d82622 2023-02-09 14:54:17,269 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing f9cd34e6dad51e106afa9855e7d82622, disabling compactions & flushes 2023-02-09 14:54:17,269 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622. 2023-02-09 14:54:17,269 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622. 2023-02-09 14:54:17,269 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622. after waiting 0 ms 2023-02-09 14:54:17,269 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622. 2023-02-09 14:54:17,269 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=10ba4d60d6f9660fb4081f6b504506d4, regionState=CLOSED 2023-02-09 14:54:17,269 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675954457269"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954457269"}]},"ts":"1675954457269"} 2023-02-09 14:54:17,270 DEBUG [StoreCloser-testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/24d455dade92499595cdab11694c97e3.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/24d455dade92499595cdab11694c97e3-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/35c28684129d4100bc06837375c9fcfc.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/35c28684129d4100bc06837375c9fcfc-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/982a4af6ef0e48139d05632e186d07ce.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/982a4af6ef0e48139d05632e186d07ce-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/340bfaba29544fd9bf87894b9a99da38.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/340bfaba29544fd9bf87894b9a99da38-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/784f58a7a9e84904ba6ee620343e6abc.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/784f58a7a9e84904ba6ee620343e6abc-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/2b649a280ad64e18817ae22baabaf44a.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/2b649a280ad64e18817ae22baabaf44a-bottom] to archive 2023-02-09 14:54:17,271 DEBUG [StoreCloser-testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:17,273 DEBUG [StoreCloser-testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/24d455dade92499595cdab11694c97e3.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/24d455dade92499595cdab11694c97e3.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,273 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-02-09 14:54:17,273 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure 10ba4d60d6f9660fb4081f6b504506d4, server=jenkins-hbase4.apache.org,41077,1675954443212 in 195 msec 2023-02-09 14:54:17,274 DEBUG [StoreCloser-testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/35c28684129d4100bc06837375c9fcfc.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/35c28684129d4100bc06837375c9fcfc.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,275 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=10ba4d60d6f9660fb4081f6b504506d4, UNASSIGN in 200 msec 2023-02-09 14:54:17,276 DEBUG [StoreCloser-testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/982a4af6ef0e48139d05632e186d07ce.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/982a4af6ef0e48139d05632e186d07ce.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,277 DEBUG [StoreCloser-testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/340bfaba29544fd9bf87894b9a99da38.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/340bfaba29544fd9bf87894b9a99da38.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,279 DEBUG [StoreCloser-testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/784f58a7a9e84904ba6ee620343e6abc.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/784f58a7a9e84904ba6ee620343e6abc.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,280 DEBUG [StoreCloser-testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,282 DEBUG [StoreCloser-testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/2b649a280ad64e18817ae22baabaf44a.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/2b649a280ad64e18817ae22baabaf44a.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,284 DEBUG [StoreCloser-testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/71f26af779b74c70842bff5a96f82839.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/71f26af779b74c70842bff5a96f82839-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/ace0880e72924e5c88c7cdf409a5ceb0.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ace0880e72924e5c88c7cdf409a5ceb0-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/d7550d6f781e465f9c9cbaa270fd52a2.2f41522f4f390d8b073e3a5237c42fff->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/d7550d6f781e465f9c9cbaa270fd52a2-bottom] to archive 2023-02-09 14:54:17,285 DEBUG [StoreCloser-testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:17,286 DEBUG [StoreCloser-testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,288 DEBUG [StoreCloser-testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/71f26af779b74c70842bff5a96f82839.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/71f26af779b74c70842bff5a96f82839.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,289 DEBUG [StoreCloser-testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/ace0880e72924e5c88c7cdf409a5ceb0.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/ace0880e72924e5c88c7cdf409a5ceb0.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,291 DEBUG [StoreCloser-testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,292 DEBUG [StoreCloser-testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,293 DEBUG [StoreCloser-testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,295 DEBUG [StoreCloser-testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/d7550d6f781e465f9c9cbaa270fd52a2.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/d7550d6f781e465f9c9cbaa270fd52a2.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,299 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 14:54:17,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:54:17,300 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622. 2023-02-09 14:54:17,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for f9cd34e6dad51e106afa9855e7d82622: Waiting for close lock at 1675954457269Running coprocessor pre-close hooks at 1675954457269Disabling compacts and flushes for region at 1675954457269Disabling writes for close at 1675954457269Writing region close event to WAL at 1675954457296 (+27 ms)Running coprocessor post-close hooks at 1675954457300 (+4 ms)Closed at 1675954457300 2023-02-09 14:54:17,302 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed f9cd34e6dad51e106afa9855e7d82622 2023-02-09 14:54:17,302 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=f9cd34e6dad51e106afa9855e7d82622, regionState=CLOSED 2023-02-09 14:54:17,302 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675954457302"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954457302"}]},"ts":"1675954457302"} 2023-02-09 14:54:17,306 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-02-09 14:54:17,306 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure f9cd34e6dad51e106afa9855e7d82622, server=jenkins-hbase4.apache.org,41077,1675954443212 in 223 msec 2023-02-09 14:54:17,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-02-09 14:54:17,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=f9cd34e6dad51e106afa9855e7d82622, UNASSIGN in 233 msec 2023-02-09 14:54:17,309 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954457309"}]},"ts":"1675954457309"} 2023-02-09 14:54:17,310 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-02-09 14:54:17,312 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-02-09 14:54:17,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 249 msec 2023-02-09 14:54:17,371 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:17,371 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:17,372 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-09 14:54:17,372 INFO [Listener at localhost/40823] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-02-09 14:54:17,372 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:17,372 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:17,373 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitWithoutPONR 2023-02-09 14:54:17,374 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:17,376 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:17,376 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:17,376 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:17,377 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:17,377 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-09 14:54:17,383 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,383 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622 2023-02-09 14:54:17,383 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:17,385 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/recovered.edits] 2023-02-09 14:54:17,386 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/recovered.edits] 2023-02-09 14:54:17,386 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/recovered.edits] 2023-02-09 14:54:17,389 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/75df77dbc1bd4662a24a9c233649296b to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf1/75df77dbc1bd4662a24a9c233649296b 2023-02-09 14:54:17,389 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/.regioninfo to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/.regioninfo 2023-02-09 14:54:17,390 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e 2023-02-09 14:54:17,392 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/24d455dade92499595cdab11694c97e3 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/24d455dade92499595cdab11694c97e3 2023-02-09 14:54:17,393 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/3530e87b21ad4cc8aa6e09c4338077f1 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/cf2/3530e87b21ad4cc8aa6e09c4338077f1 2023-02-09 14:54:17,393 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,394 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/2b649a280ad64e18817ae22baabaf44a to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/2b649a280ad64e18817ae22baabaf44a 2023-02-09 14:54:17,395 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf1/24d455dade92499595cdab11694c97e3.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf1/24d455dade92499595cdab11694c97e3.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,395 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/340bfaba29544fd9bf87894b9a99da38 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/340bfaba29544fd9bf87894b9a99da38 2023-02-09 14:54:17,396 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/.regioninfo to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/.regioninfo 2023-02-09 14:54:17,396 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf1/2b649a280ad64e18817ae22baabaf44a.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf1/2b649a280ad64e18817ae22baabaf44a.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,397 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/35c28684129d4100bc06837375c9fcfc to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/35c28684129d4100bc06837375c9fcfc 2023-02-09 14:54:17,398 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf1/340bfaba29544fd9bf87894b9a99da38.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf1/340bfaba29544fd9bf87894b9a99da38.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,398 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/784f58a7a9e84904ba6ee620343e6abc to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/784f58a7a9e84904ba6ee620343e6abc 2023-02-09 14:54:17,400 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf1/0be6e1aa2b3f4c87ab1ae4ba21138f9e.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,400 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf1/35c28684129d4100bc06837375c9fcfc.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf1/35c28684129d4100bc06837375c9fcfc.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,400 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/982a4af6ef0e48139d05632e186d07ce to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf1/982a4af6ef0e48139d05632e186d07ce 2023-02-09 14:54:17,402 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf1/24d455dade92499595cdab11694c97e3.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf1/24d455dade92499595cdab11694c97e3.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,402 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf1/784f58a7a9e84904ba6ee620343e6abc.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf1/784f58a7a9e84904ba6ee620343e6abc.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,403 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf1/2b649a280ad64e18817ae22baabaf44a.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf1/2b649a280ad64e18817ae22baabaf44a.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,404 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf1/982a4af6ef0e48139d05632e186d07ce.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf1/982a4af6ef0e48139d05632e186d07ce.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,404 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0 2023-02-09 14:54:17,405 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf1/340bfaba29544fd9bf87894b9a99da38.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf1/340bfaba29544fd9bf87894b9a99da38.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,406 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/71f26af779b74c70842bff5a96f82839 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/71f26af779b74c70842bff5a96f82839 2023-02-09 14:54:17,407 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf1/35c28684129d4100bc06837375c9fcfc.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf1/35c28684129d4100bc06837375c9fcfc.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,408 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,409 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ace0880e72924e5c88c7cdf409a5ceb0 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ace0880e72924e5c88c7cdf409a5ceb0 2023-02-09 14:54:17,409 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf1/784f58a7a9e84904ba6ee620343e6abc.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf1/784f58a7a9e84904ba6ee620343e6abc.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,411 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf2/71f26af779b74c70842bff5a96f82839.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf2/71f26af779b74c70842bff5a96f82839.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,411 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d 2023-02-09 14:54:17,411 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf1/982a4af6ef0e48139d05632e186d07ce.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf1/982a4af6ef0e48139d05632e186d07ce.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,413 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf2/ace0880e72924e5c88c7cdf409a5ceb0.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf2/ace0880e72924e5c88c7cdf409a5ceb0.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,413 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab 2023-02-09 14:54:17,414 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,415 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/d7550d6f781e465f9c9cbaa270fd52a2 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/d7550d6f781e465f9c9cbaa270fd52a2 2023-02-09 14:54:17,416 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf2/46fbacd2dcbc4ea1b04e7a1b227349b0.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,416 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,417 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106 2023-02-09 14:54:17,418 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf2/71f26af779b74c70842bff5a96f82839.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf2/71f26af779b74c70842bff5a96f82839.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,418 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf2/d7550d6f781e465f9c9cbaa270fd52a2.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf2/d7550d6f781e465f9c9cbaa270fd52a2.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,420 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf2/ace0880e72924e5c88c7cdf409a5ceb0.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf2/ace0880e72924e5c88c7cdf409a5ceb0.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,420 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/10ba4d60d6f9660fb4081f6b504506d4/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,421 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/recovered.edits/85.seqid to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff/recovered.edits/85.seqid 2023-02-09 14:54:17,421 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf2/b03ae9b135764b5f9c6b0a3c5f96de5d.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,422 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,423 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf2/ca9642f1f44d4e39b488e2e8fa97f0ab.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,423 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/4eb79868f33d4bf09a1498d56814c7cc to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf1/4eb79868f33d4bf09a1498d56814c7cc 2023-02-09 14:54:17,425 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf2/d7550d6f781e465f9c9cbaa270fd52a2.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf2/d7550d6f781e465f9c9cbaa270fd52a2.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,426 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/41d13af2f65e41f5b42af331a3a4af4a to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/cf2/41d13af2f65e41f5b42af331a3a4af4a 2023-02-09 14:54:17,427 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106.2f41522f4f390d8b073e3a5237c42fff to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/f9cd34e6dad51e106afa9855e7d82622/cf2/f9de83d1b2674c6bbda0f8d1b1ec7106.2f41522f4f390d8b073e3a5237c42fff 2023-02-09 14:54:17,430 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/recovered.edits/90.seqid to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4/recovered.edits/90.seqid 2023-02-09 14:54:17,433 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/recovered.edits/90.seqid to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622/recovered.edits/90.seqid 2023-02-09 14:54:17,434 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:17,434 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitWithoutPONR/f9cd34e6dad51e106afa9855e7d82622 2023-02-09 14:54:17,434 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-09 14:54:17,437 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:17,439 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-02-09 14:54:17,442 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-02-09 14:54:17,443 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:17,443 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-02-09 14:54:17,443 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954457443"}]},"ts":"9223372036854775807"} 2023-02-09 14:54:17,443 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954457443"}]},"ts":"9223372036854775807"} 2023-02-09 14:54:17,443 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954457443"}]},"ts":"9223372036854775807"} 2023-02-09 14:54:17,445 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 14:54:17,445 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 2f41522f4f390d8b073e3a5237c42fff, NAME => 'testSplitWithoutPONR,,1675954451258.2f41522f4f390d8b073e3a5237c42fff.', STARTKEY => '', ENDKEY => ''}, {ENCODED => f9cd34e6dad51e106afa9855e7d82622, NAME => 'testSplitWithoutPONR,,1675954453262.f9cd34e6dad51e106afa9855e7d82622.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 10ba4d60d6f9660fb4081f6b504506d4, NAME => 'testSplitWithoutPONR,41,1675954453262.10ba4d60d6f9660fb4081f6b504506d4.', STARTKEY => '41', ENDKEY => ''}] 2023-02-09 14:54:17,445 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-02-09 14:54:17,445 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675954457445"}]},"ts":"9223372036854775807"} 2023-02-09 14:54:17,447 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-02-09 14:54:17,449 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:17,450 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 76 msec 2023-02-09 14:54:17,477 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:17,478 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:17,479 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-09 14:54:17,479 INFO [Listener at localhost/40823] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-02-09 14:54:17,500 INFO [Listener at localhost/40823] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=401 (was 392) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:54468 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:49780 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase4:41077-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/cluster_fa49ce5e-dcbf-0fd7-6b70-a45a30f7cf80/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:54430 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:37622 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:49714 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:37640 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:37024 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:49790 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/cluster_fa49ce5e-dcbf-0fd7-6b70-a45a30f7cf80/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:36712 [Waiting for operation #27] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x640c2dac-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:37646 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:54438 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:54412 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:36698 [Waiting for operation #26] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:54416 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:49752 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/cluster_fa49ce5e-dcbf-0fd7-6b70-a45a30f7cf80/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/cluster_fa49ce5e-dcbf-0fd7-6b70-a45a30f7cf80/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:37030 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=855 (was 844) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=281 (was 305), ProcessCount=168 (was 168), AvailableMemoryMB=3241 (was 3442) 2023-02-09 14:54:17,515 INFO [Listener at localhost/40823] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=401, OpenFileDescriptor=855, MaxFileDescriptor=60000, SystemLoadAverage=281, ProcessCount=168, AvailableMemoryMB=3240 2023-02-09 14:54:17,516 WARN [Listener at localhost/40823] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 14:54:17,516 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:17,516 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:17,519 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 14:54:17,519 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 14:54:17,521 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:54:17,523 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 14:54:17,524 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 14:54:17,525 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:17,526 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b empty. 2023-02-09 14:54:17,526 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:17,526 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-09 14:54:17,536 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-09 14:54:17,538 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 458ec544613af51e1d69337b4a7fa39b, NAME => 'testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp 2023-02-09 14:54:17,545 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:17,545 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 458ec544613af51e1d69337b4a7fa39b, disabling compactions & flushes 2023-02-09 14:54:17,545 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:17,545 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:17,545 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. after waiting 0 ms 2023-02-09 14:54:17,546 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:17,546 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:17,546 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 458ec544613af51e1d69337b4a7fa39b: Waiting for close lock at 1675954457545Disabling compacts and flushes for region at 1675954457545Disabling writes for close at 1675954457546 (+1 ms)Writing region close event to WAL at 1675954457546Closed at 1675954457546 2023-02-09 14:54:17,548 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 14:54:17,549 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675954457549"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954457549"}]},"ts":"1675954457549"} 2023-02-09 14:54:17,550 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 14:54:17,551 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 14:54:17,551 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954457551"}]},"ts":"1675954457551"} 2023-02-09 14:54:17,553 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-09 14:54:17,556 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 14:54:17,557 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 14:54:17,557 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 14:54:17,557 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 14:54:17,557 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 14:54:17,557 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN}] 2023-02-09 14:54:17,559 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN 2023-02-09 14:54:17,559 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41077,1675954443212; forceNewPlan=false, retain=false 2023-02-09 14:54:17,710 INFO [jenkins-hbase4:43829] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 14:54:17,711 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=458ec544613af51e1d69337b4a7fa39b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:17,711 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675954457711"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954457711"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954457711"}]},"ts":"1675954457711"} 2023-02-09 14:54:17,713 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212}] 2023-02-09 14:54:17,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:17,868 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 458ec544613af51e1d69337b4a7fa39b, NAME => 'testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:54:17,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:54:17,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:17,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:17,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:17,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:17,871 INFO [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:17,872 DEBUG [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1 2023-02-09 14:54:17,872 DEBUG [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1 2023-02-09 14:54:17,872 INFO [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 458ec544613af51e1d69337b4a7fa39b columnFamilyName cf1 2023-02-09 14:54:17,873 INFO [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] regionserver.HStore(310): Store=458ec544613af51e1d69337b4a7fa39b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:17,873 INFO [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:17,874 DEBUG [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2 2023-02-09 14:54:17,874 DEBUG [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2 2023-02-09 14:54:17,875 INFO [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 458ec544613af51e1d69337b4a7fa39b columnFamilyName cf2 2023-02-09 14:54:17,875 INFO [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] regionserver.HStore(310): Store=458ec544613af51e1d69337b4a7fa39b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:17,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:17,876 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:17,878 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:54:17,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:17,881 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:54:17,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 458ec544613af51e1d69337b4a7fa39b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61975272, jitterRate=-0.07649648189544678}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:54:17,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 458ec544613af51e1d69337b4a7fa39b: Running coprocessor pre-open hook at 1675954457870Writing region info on filesystem at 1675954457870Initializing all the Stores at 1675954457870Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954457870Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954457871 (+1 ms)Cleaning up temporary data from old regions at 1675954457877 (+6 ms)Cleaning up detritus from prior splits at 1675954457877Running coprocessor post-open hooks at 1675954457882 (+5 ms)Region opened successfully at 1675954457882 2023-02-09 14:54:17,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b., pid=43, masterSystemTime=1675954457865 2023-02-09 14:54:17,884 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:17,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:17,885 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=458ec544613af51e1d69337b4a7fa39b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:17,885 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675954457885"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954457885"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954457885"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954457885"}]},"ts":"1675954457885"} 2023-02-09 14:54:17,888 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-02-09 14:54:17,888 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 in 173 msec 2023-02-09 14:54:17,890 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-02-09 14:54:17,890 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN in 331 msec 2023-02-09 14:54:17,891 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 14:54:17,891 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954457891"}]},"ts":"1675954457891"} 2023-02-09 14:54:17,893 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-09 14:54:17,904 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 14:54:17,905 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 384 msec 2023-02-09 14:54:18,031 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:18,031 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:18,032 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:18,034 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-09 14:54:18,034 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-09 14:54:18,034 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:18,034 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-09 14:54:18,034 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 14:54:18,035 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,035 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,036 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,036 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,036 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,036 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,036 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,036 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:18,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,038 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,038 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,038 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:18,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:18,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,038 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 14:54:18,038 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 14:54:18,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:18,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,038 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 14:54:18,038 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 14:54:18,039 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 14:54:18,039 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 14:54:18,039 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 14:54:18,039 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 14:54:18,039 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 14:54:18,039 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:18,039 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:18,039 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:18,039 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:54:18,039 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 14:54:18,039 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:54:18,039 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 14:54:18,039 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 14:54:18,039 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 14:54:18,039 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:54:18,039 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:18,039 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:18,039 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:18,039 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. started... 2023-02-09 14:54:18,040 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 458ec544613af51e1d69337b4a7fa39b 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 14:54:18,041 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,043 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,043 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:18,044 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,044 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,044 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:18,058 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf1/b8935f1f3a244c18b4524f10dc75d9fa 2023-02-09 14:54:18,071 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf2/cf86fb5842b54944b820c99c213e667b 2023-02-09 14:54:18,076 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf1/b8935f1f3a244c18b4524f10dc75d9fa as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/b8935f1f3a244c18b4524f10dc75d9fa 2023-02-09 14:54:18,080 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/b8935f1f3a244c18b4524f10dc75d9fa, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 14:54:18,080 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf2/cf86fb5842b54944b820c99c213e667b as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/cf86fb5842b54944b820c99c213e667b 2023-02-09 14:54:18,084 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/cf86fb5842b54944b820c99c213e667b, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 14:54:18,085 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 458ec544613af51e1d69337b4a7fa39b in 45ms, sequenceid=6, compaction requested=false 2023-02-09 14:54:18,085 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-02-09 14:54:18,086 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 458ec544613af51e1d69337b4a7fa39b: 2023-02-09 14:54:18,086 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:18,086 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:18,086 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:18,086 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,086 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 14:54:18,086 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 14:54:18,088 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,088 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:18,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:18,088 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,088 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:18,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:18,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:18,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:18,090 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-09 14:54:18,090 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 14:54:18,090 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7bcc06b6[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:18,090 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,091 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,091 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,091 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,092 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,092 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:18,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,092 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:54:18,092 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:54:18,092 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 14:54:18,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,092 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:18,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,092 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 14:54:18,092 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:54:18,092 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:18,092 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 14:54:18,092 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:18,095 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 14:54:18,096 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:18,096 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 14:54:18,096 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 14:54:18,096 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,096 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 14:54:18,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:18,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:18,096 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:18,096 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:18,096 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 14:54:18,096 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 14:54:18,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:18,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:18,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:18,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,41077,1675954443212': 2023-02-09 14:54:18,099 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 14:54:18,099 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-09 14:54:18,099 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:18,100 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:18,100 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-09 14:54:18,100 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:18,102 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,102 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,102 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,102 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,102 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,102 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,103 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:18,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:18,103 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,103 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,103 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:18,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,104 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,104 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:18,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,105 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,106 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:18,106 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,109 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,109 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,109 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,109 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,109 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,109 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,109 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,109 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-09 14:54:18,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,110 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:18,109 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,109 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,109 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:18,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,110 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,110 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:18,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,110 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:18,110 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,111 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,111 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,111 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,111 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,111 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,111 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,211 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:18,211 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:18,211 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:18,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:18,227 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:18,227 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:18,228 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:18,229 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:18,229 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-09 14:54:18,229 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-09 14:54:18,229 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:18,230 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-09 14:54:18,230 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 14:54:18,230 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,230 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,231 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,231 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,231 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,231 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,232 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,232 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:18,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,232 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,232 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,232 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:18,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:18,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:18,233 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 14:54:18,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,233 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 14:54:18,233 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 14:54:18,233 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 14:54:18,233 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 14:54:18,233 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 14:54:18,233 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 14:54:18,234 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:18,234 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 14:54:18,233 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 14:54:18,234 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 14:54:18,234 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 14:54:18,234 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:54:18,234 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 14:54:18,234 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 14:54:18,234 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:18,234 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 14:54:18,234 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:54:18,234 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:18,234 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:54:18,234 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 14:54:18,234 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 14:54:18,234 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:18,234 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:18,234 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:18,234 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. started... 2023-02-09 14:54:18,235 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 458ec544613af51e1d69337b4a7fa39b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:54:18,235 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,237 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,237 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,237 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:18,242 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,242 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:18,251 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf1/b857c7123fc04f1fb52a1b13bef9712a 2023-02-09 14:54:18,266 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf2/ff6b579b48cb4d36b866bf386f921a7f 2023-02-09 14:54:18,271 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf1/b857c7123fc04f1fb52a1b13bef9712a as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/b857c7123fc04f1fb52a1b13bef9712a 2023-02-09 14:54:18,275 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/b857c7123fc04f1fb52a1b13bef9712a, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 14:54:18,276 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf2/ff6b579b48cb4d36b866bf386f921a7f as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/ff6b579b48cb4d36b866bf386f921a7f 2023-02-09 14:54:18,280 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/ff6b579b48cb4d36b866bf386f921a7f, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 14:54:18,281 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 458ec544613af51e1d69337b4a7fa39b in 47ms, sequenceid=19, compaction requested=false 2023-02-09 14:54:18,281 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 458ec544613af51e1d69337b4a7fa39b: 2023-02-09 14:54:18,281 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:18,281 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:18,281 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:18,282 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,282 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 14:54:18,282 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 14:54:18,283 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,283 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:18,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:18,283 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,283 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:18,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:18,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:18,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:18,286 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-09 14:54:18,286 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4320fdf3[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:18,286 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 14:54:18,286 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,287 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,287 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,287 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,287 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:54:18,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,287 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,287 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:18,287 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 14:54:18,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,287 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:18,287 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:54:18,287 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 14:54:18,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,287 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:18,287 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:54:18,287 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 14:54:18,287 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:18,288 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,288 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 14:54:18,289 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:18,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:18,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:18,289 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 14:54:18,292 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 14:54:18,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:18,292 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:18,292 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 14:54:18,292 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 14:54:18,293 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:18,293 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 14:54:18,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:18,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:18,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,41077,1675954443212': 2023-02-09 14:54:18,296 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 14:54:18,296 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-09 14:54:18,296 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:18,296 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:18,296 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-09 14:54:18,296 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:18,297 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,297 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:18,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:18,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,297 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,297 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,297 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:18,297 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,297 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,298 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,298 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:18,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,298 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:18,300 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,300 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,300 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,303 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,303 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,303 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,303 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,303 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,303 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:18,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,303 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,303 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-09 14:54:18,303 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:18,303 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,303 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,303 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,303 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:18,303 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,303 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:18,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,304 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,304 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,304 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,304 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,304 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,304 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,304 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,404 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:18,404 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:18,404 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:18,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:18,419 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:18,419 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:18,420 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:18,422 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:18,422 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-09 14:54:18,422 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-09 14:54:18,422 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:18,422 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-09 14:54:18,422 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 14:54:18,423 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,423 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,425 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,425 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,425 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,425 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,425 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,426 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:18,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,427 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,427 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:18,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,427 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 14:54:18,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:18,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,427 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 14:54:18,427 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,428 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 14:54:18,428 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 14:54:18,428 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 14:54:18,428 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:18,428 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 14:54:18,428 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:54:18,428 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 14:54:18,428 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 14:54:18,428 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:18,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:18,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,428 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:54:18,428 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 14:54:18,428 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 14:54:18,428 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 14:54:18,429 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:18,429 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:18,429 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:18,429 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 14:54:18,429 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. started... 2023-02-09 14:54:18,429 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 14:54:18,429 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 14:54:18,429 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:18,429 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 458ec544613af51e1d69337b4a7fa39b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:54:18,429 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:54:18,429 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 14:54:18,429 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 14:54:18,431 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,431 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,431 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:18,441 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,441 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,441 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:18,455 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf1/0b22449ce5bb43f8a79295863be22e8e 2023-02-09 14:54:18,470 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf2/b5c988f26128498a84dd733de608ff49 2023-02-09 14:54:18,475 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf1/0b22449ce5bb43f8a79295863be22e8e as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/0b22449ce5bb43f8a79295863be22e8e 2023-02-09 14:54:18,479 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/0b22449ce5bb43f8a79295863be22e8e, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 14:54:18,480 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf2/b5c988f26128498a84dd733de608ff49 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/b5c988f26128498a84dd733de608ff49 2023-02-09 14:54:18,484 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/b5c988f26128498a84dd733de608ff49, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 14:54:18,485 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 458ec544613af51e1d69337b4a7fa39b in 56ms, sequenceid=32, compaction requested=true 2023-02-09 14:54:18,485 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 458ec544613af51e1d69337b4a7fa39b: 2023-02-09 14:54:18,485 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:18,485 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:18,485 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:18,485 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,485 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 14:54:18,485 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 14:54:18,487 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,487 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:18,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:18,487 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,487 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:18,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:18,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:18,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:18,489 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-09 14:54:18,489 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@31ec0e62[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:18,489 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 14:54:18,489 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,491 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,491 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,491 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,491 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:54:18,491 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 14:54:18,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,491 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:18,491 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:54:18,491 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 14:54:18,491 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:18,491 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,491 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:18,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,491 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:54:18,491 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 14:54:18,492 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:18,492 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,493 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 14:54:18,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,493 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:18,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:18,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:18,493 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 14:54:18,493 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 14:54:18,493 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:18,493 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 14:54:18,493 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 14:54:18,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:18,493 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:18,494 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 14:54:18,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:18,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:18,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,41077,1675954443212': 2023-02-09 14:54:18,496 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 14:54:18,496 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-09 14:54:18,496 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:18,496 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:18,496 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-09 14:54:18,496 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:18,498 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,498 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,498 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,498 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,498 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,498 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,498 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:18,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:18,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,499 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:18,499 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,499 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:18,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,501 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:18,501 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,501 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,507 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,507 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,507 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,508 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,508 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:18,507 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,507 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,508 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,508 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-09 14:54:18,508 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:18,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,508 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,508 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,508 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,508 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:18,508 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,508 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:18,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,508 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,509 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,509 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,509 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,509 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,509 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,609 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:18,609 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:18,609 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:18,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:18,625 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:18,626 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:18,626 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:18,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:18,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-09 14:54:18,628 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-09 14:54:18,628 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:18,629 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-09 14:54:18,629 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 14:54:18,629 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,629 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,630 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,630 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,630 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,630 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,631 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,631 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:18,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,631 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,631 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,631 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:18,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:18,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,631 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 14:54:18,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:18,632 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 14:54:18,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,632 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 14:54:18,632 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 14:54:18,632 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 14:54:18,632 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 14:54:18,632 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 14:54:18,632 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 14:54:18,632 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:18,632 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 14:54:18,632 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 14:54:18,632 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 14:54:18,632 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 14:54:18,632 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:54:18,632 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:18,632 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:18,632 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:54:18,632 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 14:54:18,632 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:18,632 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:54:18,632 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:18,632 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:18,632 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 14:54:18,633 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. started... 2023-02-09 14:54:18,632 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 14:54:18,633 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 14:54:18,633 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 458ec544613af51e1d69337b4a7fa39b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:54:18,634 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,634 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,634 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,634 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:18,634 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,634 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:18,642 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf1/04db8ad2f1be4c56b4710dcb46341919 2023-02-09 14:54:18,654 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf2/1bedbf1b41144837aee62f61a6d624d0 2023-02-09 14:54:18,658 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf1/04db8ad2f1be4c56b4710dcb46341919 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/04db8ad2f1be4c56b4710dcb46341919 2023-02-09 14:54:18,662 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/04db8ad2f1be4c56b4710dcb46341919, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 14:54:18,663 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf2/1bedbf1b41144837aee62f61a6d624d0 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/1bedbf1b41144837aee62f61a6d624d0 2023-02-09 14:54:18,667 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/1bedbf1b41144837aee62f61a6d624d0, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 14:54:18,668 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 458ec544613af51e1d69337b4a7fa39b in 35ms, sequenceid=45, compaction requested=true 2023-02-09 14:54:18,668 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 458ec544613af51e1d69337b4a7fa39b: 2023-02-09 14:54:18,668 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:18,668 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:18,668 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:18,668 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,668 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 14:54:18,669 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 14:54:18,671 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,671 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:18,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:18,672 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,672 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:18,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:18,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:18,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:18,674 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-09 14:54:18,674 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@19c400f3[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:18,674 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 14:54:18,674 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,675 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,675 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,675 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,675 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,675 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:54:18,675 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 14:54:18,675 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:18,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,675 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:18,675 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:54:18,675 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 14:54:18,675 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:18,676 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,676 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:54:18,676 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 14:54:18,676 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:18,677 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 14:54:18,677 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 14:54:18,677 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:18,677 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,677 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 14:54:18,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:18,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:18,677 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:18,677 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 14:54:18,677 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 14:54:18,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:18,677 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:18,678 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 14:54:18,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:18,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:18,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,41077,1675954443212': 2023-02-09 14:54:18,680 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 14:54:18,680 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-09 14:54:18,680 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:18,680 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:18,681 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-09 14:54:18,681 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:18,688 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,688 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,688 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,688 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,688 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:18,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:18,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,688 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,688 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:18,688 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,689 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:18,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,689 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:18,691 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,691 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,691 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,693 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,693 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,693 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,693 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,693 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,693 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,694 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,693 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,694 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-09 14:54:18,694 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:18,694 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:18,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,693 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,694 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,694 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,694 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,694 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:18,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,694 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,695 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,695 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:18,695 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,695 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,695 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,795 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:18,795 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:18,795 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:18,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:18,810 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:18,811 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:18,812 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:18,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:18,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-09 14:54:18,813 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-09 14:54:18,813 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:18,813 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-09 14:54:18,813 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 14:54:18,814 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,814 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,815 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,815 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,815 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,816 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,816 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:18,815 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,816 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,816 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:18,816 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,817 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 14:54:18,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:18,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:18,817 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 14:54:18,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,817 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 14:54:18,817 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 14:54:18,817 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 14:54:18,817 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:18,817 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 14:54:18,817 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:54:18,817 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 14:54:18,817 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 14:54:18,817 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 14:54:18,817 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 14:54:18,817 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 14:54:18,817 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:18,817 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:54:18,817 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 14:54:18,817 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 14:54:18,818 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 14:54:18,818 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 14:54:18,818 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 14:54:18,818 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:18,818 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:54:18,818 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:18,818 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:18,818 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:18,818 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. started... 2023-02-09 14:54:18,819 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 458ec544613af51e1d69337b4a7fa39b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:54:18,819 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,819 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,820 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,820 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:18,820 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,820 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:18,832 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf1/92085b16bd114633a8a9c119de934ddd 2023-02-09 14:54:18,845 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf2/3eeee066fc07454b89a5897742cca0db 2023-02-09 14:54:18,849 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf1/92085b16bd114633a8a9c119de934ddd as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/92085b16bd114633a8a9c119de934ddd 2023-02-09 14:54:18,853 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/92085b16bd114633a8a9c119de934ddd, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 14:54:18,854 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf2/3eeee066fc07454b89a5897742cca0db as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/3eeee066fc07454b89a5897742cca0db 2023-02-09 14:54:18,858 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/3eeee066fc07454b89a5897742cca0db, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 14:54:18,859 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 458ec544613af51e1d69337b4a7fa39b in 41ms, sequenceid=58, compaction requested=true 2023-02-09 14:54:18,859 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 458ec544613af51e1d69337b4a7fa39b: 2023-02-09 14:54:18,859 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:18,859 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:18,859 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:18,859 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,859 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 14:54:18,859 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 14:54:18,862 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,862 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:18,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:18,862 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,862 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:18,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:18,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:18,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:18,864 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-09 14:54:18,864 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2a65ccb[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:18,864 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 14:54:18,864 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,865 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,865 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,865 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,866 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,866 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:54:18,866 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 14:54:18,866 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:18,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,866 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:18,866 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:54:18,866 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 14:54:18,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,866 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:18,866 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:54:18,866 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 14:54:18,866 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:18,867 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 14:54:18,867 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:18,867 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 14:54:18,873 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 14:54:18,873 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,873 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 14:54:18,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:18,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:18,873 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:18,873 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:18,873 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 14:54:18,873 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 14:54:18,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:18,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:18,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:18,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,41077,1675954443212': 2023-02-09 14:54:18,876 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 14:54:18,876 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-09 14:54:18,876 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:18,876 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:18,876 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-09 14:54:18,876 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:18,878 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,878 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,878 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,878 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,878 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,878 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,878 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:18,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:18,878 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:18,879 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,879 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:18,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,881 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:18,881 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:18,881 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,884 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,884 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,884 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,884 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,884 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,884 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,884 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-09 14:54:18,884 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:18,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,884 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:18,884 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,884 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:18,884 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,884 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:18,885 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:18,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,885 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:18,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:18,885 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:18,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:18,885 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,885 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:18,885 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:18,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:18,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:18,885 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:18,885 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:18,886 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:18,985 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:18,986 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:18,986 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:18,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:19,000 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:19,000 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:19,001 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:19,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:19,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-09 14:54:19,002 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-09 14:54:19,003 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:19,003 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-09 14:54:19,003 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 14:54:19,003 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:19,003 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:19,004 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:19,004 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:19,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:19,004 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:19,004 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:19,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:19,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:19,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:19,005 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:19,005 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:19,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:19,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:19,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:19,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:19,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:19,005 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:19,005 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:19,005 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:19,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:19,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:19,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:19,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:19,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:19,006 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 14:54:19,006 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 14:54:19,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:19,006 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-09 14:54:19,006 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 14:54:19,006 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 14:54:19,006 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 14:54:19,006 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 14:54:19,006 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 14:54:19,006 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-09 14:54:19,006 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 14:54:19,006 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-09 14:54:19,006 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:19,006 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-09 14:54:19,006 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:54:19,006 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:19,006 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:19,006 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:54:19,006 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:54:19,007 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 14:54:19,006 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 14:54:19,007 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 14:54:19,007 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 14:54:19,007 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:19,007 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:19,007 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:19,007 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. started... 2023-02-09 14:54:19,007 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 458ec544613af51e1d69337b4a7fa39b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:54:19,009 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:19,009 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:19,010 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:19,010 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:19,010 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:19,010 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:19,016 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf1/999689ceaa644f9db0bff84d94a40860 2023-02-09 14:54:19,028 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf2/b6dfbc2187574a54ad15bb8661b211f4 2023-02-09 14:54:19,033 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf1/999689ceaa644f9db0bff84d94a40860 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/999689ceaa644f9db0bff84d94a40860 2023-02-09 14:54:19,037 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/999689ceaa644f9db0bff84d94a40860, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 14:54:19,038 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf2/b6dfbc2187574a54ad15bb8661b211f4 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/b6dfbc2187574a54ad15bb8661b211f4 2023-02-09 14:54:19,043 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/b6dfbc2187574a54ad15bb8661b211f4, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 14:54:19,044 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 458ec544613af51e1d69337b4a7fa39b in 37ms, sequenceid=71, compaction requested=true 2023-02-09 14:54:19,044 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 458ec544613af51e1d69337b4a7fa39b: 2023-02-09 14:54:19,044 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:19,044 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:19,044 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:19,044 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:19,044 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-09 14:54:19,044 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-09 14:54:19,046 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:19,046 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:19,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:19,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:19,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:19,046 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:19,046 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:19,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:19,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:19,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:19,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:19,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:19,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:19,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:19,048 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-09 14:54:19,048 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@462df609[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:19,048 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 14:54:19,048 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:19,049 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:19,049 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:19,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:19,049 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:19,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:19,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:19,049 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:54:19,049 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 14:54:19,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:19,049 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:19,049 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:19,049 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:19,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:19,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:19,049 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:54:19,049 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:54:19,050 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 14:54:19,049 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-09 14:54:19,050 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:19,050 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:19,051 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:19,051 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 14:54:19,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:19,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:19,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:19,051 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:19,051 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 14:54:19,052 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 14:54:19,052 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-09 14:54:19,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:19,052 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:19,053 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:19,053 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 14:54:19,053 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-09 14:54:19,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:19,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:19,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:19,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:19,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:19,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:19,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:19,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:19,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:19,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:19,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,41077,1675954443212': 2023-02-09 14:54:19,055 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 14:54:19,055 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-09 14:54:19,055 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:19,055 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:19,055 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-09 14:54:19,055 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:19,057 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:19,057 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:19,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:19,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:19,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:19,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:19,057 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:19,057 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:19,057 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:19,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:19,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:19,057 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:19,057 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:19,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:19,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:19,057 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:19,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:19,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:19,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:19,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:19,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:19,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:19,058 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:19,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:19,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:19,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:19,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:19,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:19,058 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:19,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:19,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:19,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:19,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:19,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:19,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:19,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:19,060 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:19,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-09 14:54:19,060 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:19,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:19,060 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:19,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:19,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:19,065 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:19,065 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:19,065 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:19,065 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:19,065 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:19,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:19,065 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-09 14:54:19,065 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:19,065 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:19,065 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:19,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:19,065 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:19,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:19,065 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:19,065 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:19,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:19,066 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:19,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:19,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:19,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:19,065 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:19,066 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:19,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:19,066 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-09 14:54:19,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:19,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:19,066 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:19,066 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:19,066 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:19,066 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:19,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:19,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:19,066 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-09 14:54:19,066 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-09 14:54:19,166 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:19,166 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:19,166 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:19,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:19,428 WARN [Listener at localhost/40823] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-09 14:54:19,428 DEBUG [Listener at localhost/40823] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:19,430 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:19,431 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:19,431 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:19,431 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:19,432 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:19,432 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:19,432 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:19,432 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:19,680 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:19,680 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:19,680 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:19,680 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:19,680 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-02-09 14:54:19,681 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:19,681 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:19,681 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:19,681 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:19,682 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:19,683 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-09 14:54:19,683 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:19,683 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:19,684 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:19,684 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:19,684 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:19,684 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:19,684 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:19,685 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:19,685 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:19,685 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:19,685 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:19,685 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:19,686 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:54:19,686 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:19,686 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:19,686 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:19,686 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:19,686 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:19,687 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 14:54:19,687 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:19,688 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:19,688 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954489688 2023-02-09 14:54:19,688 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:19,690 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:19,692 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:19,692 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=458ec544613af51e1d69337b4a7fa39b, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:19,692 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:19,692 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954519692 2023-02-09 14:54:19,692 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954579692 2023-02-09 14:54:19,692 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 14:54:19,693 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:54:19,696 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:19,696 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:19,696 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:19,696 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:19,943 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:19,943 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:19,943 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:19,943 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:19,944 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:54:19,944 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:19,944 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:19,944 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:19,944 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:19,945 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:19,946 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-09 14:54:19,947 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:19,947 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:19,947 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:19,948 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:19,948 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:19,948 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:19,949 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:19,949 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:19,949 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:19,950 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:19,950 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:19,950 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:19,950 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:54:19,950 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:19,951 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:19,951 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:19,951 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:19,951 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:19,951 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 14:54:19,951 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:19,952 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:19,952 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954489952 2023-02-09 14:54:19,952 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:19,954 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:19,956 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:19,956 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=458ec544613af51e1d69337b4a7fa39b, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:19,956 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:19,956 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954519956 2023-02-09 14:54:19,956 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954579956 2023-02-09 14:54:19,956 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 14:54:19,956 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:54:19,958 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN}] 2023-02-09 14:54:19,958 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:19,958 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:19,958 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:19,958 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:20,206 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:20,207 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:20,207 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:20,207 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:20,207 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:54:20,207 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:20,207 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:20,207 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:20,207 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:20,208 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:20,209 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-09 14:54:20,209 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:20,210 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:20,210 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:20,210 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:20,210 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:20,210 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:20,211 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:20,211 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:20,211 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:20,211 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:20,212 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:20,212 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:20,212 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:54:20,213 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:20,213 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:20,213 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:20,213 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:20,213 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:20,213 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 14:54:20,214 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:20,214 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:20,214 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954490214 2023-02-09 14:54:20,215 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:20,216 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:20,218 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:20,218 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=458ec544613af51e1d69337b4a7fa39b, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:20,218 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:20,218 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954520218 2023-02-09 14:54:20,218 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954580218 2023-02-09 14:54:20,218 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 14:54:20,219 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN}] 2023-02-09 14:54:20,219 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:54:20,221 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN 2023-02-09 14:54:20,221 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=458ec544613af51e1d69337b4a7fa39b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:20,222 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675954460221"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954460221"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954460221"}]},"ts":"1675954460221"} 2023-02-09 14:54:20,223 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212}] 2023-02-09 14:54:20,223 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:20,223 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN 2023-02-09 14:54:20,223 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:20,223 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:20,469 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:20,469 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:20,469 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:20,469 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:20,471 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:54:20,472 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:20,472 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:20,472 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:20,472 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:20,473 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:20,474 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 14:54:20,474 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-09 14:54:20,475 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:20,475 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:20,475 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:20,475 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:20,476 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:20,476 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:20,476 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:20,476 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:20,477 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:20,477 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:20,477 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:20,478 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:20,478 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:54:20,479 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:20,479 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN 2023-02-09 14:54:20,479 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:20,479 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:20,479 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:20,479 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:20,479 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:20,479 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN 2023-02-09 14:54:20,479 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 14:54:20,480 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:20,481 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:20,481 INFO [Listener at localhost/40823] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b to restore RIT 2023-02-09 14:54:20,481 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954490481 2023-02-09 14:54:20,482 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN 2023-02-09 14:54:20,482 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:20,485 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:20,485 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=458ec544613af51e1d69337b4a7fa39b, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:20,485 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:20,485 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954520485 2023-02-09 14:54:20,485 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954580485 2023-02-09 14:54:20,485 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 14:54:20,486 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=458ec544613af51e1d69337b4a7fa39b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:20,486 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675954460486"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954460486"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954460486"}]},"ts":"1675954460486"} 2023-02-09 14:54:20,488 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212}] 2023-02-09 14:54:20,488 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:54:20,641 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:20,641 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 458ec544613af51e1d69337b4a7fa39b, disabling compactions & flushes 2023-02-09 14:54:20,641 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:20,641 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:20,641 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. after waiting 0 ms 2023-02-09 14:54:20,641 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:20,641 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 458ec544613af51e1d69337b4a7fa39b 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 14:54:20,651 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf1/4e549ea7cb5040f2a1f0f8141f806241 2023-02-09 14:54:20,663 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf2/f1b8c01795c5490683a21355e270af0a 2023-02-09 14:54:20,666 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf1/4e549ea7cb5040f2a1f0f8141f806241 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/4e549ea7cb5040f2a1f0f8141f806241 2023-02-09 14:54:20,670 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/4e549ea7cb5040f2a1f0f8141f806241, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 14:54:20,671 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf2/f1b8c01795c5490683a21355e270af0a as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/f1b8c01795c5490683a21355e270af0a 2023-02-09 14:54:20,675 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/f1b8c01795c5490683a21355e270af0a, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 14:54:20,676 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 458ec544613af51e1d69337b4a7fa39b in 35ms, sequenceid=82, compaction requested=true 2023-02-09 14:54:20,682 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 14:54:20,682 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:54:20,682 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:20,682 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 458ec544613af51e1d69337b4a7fa39b: Waiting for close lock at 1675954460641Running coprocessor pre-close hooks at 1675954460641Disabling compacts and flushes for region at 1675954460641Disabling writes for close at 1675954460641Obtaining lock to block concurrent updates at 1675954460641Preparing flush snapshotting stores in 458ec544613af51e1d69337b4a7fa39b at 1675954460641Finished memstore snapshotting testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675954460641Flushing stores of testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. at 1675954460642 (+1 ms)Flushing 458ec544613af51e1d69337b4a7fa39b/cf1: creating writer at 1675954460643 (+1 ms)Flushing 458ec544613af51e1d69337b4a7fa39b/cf1: appending metadata at 1675954460645 (+2 ms)Flushing 458ec544613af51e1d69337b4a7fa39b/cf1: closing flushed file at 1675954460645Flushing 458ec544613af51e1d69337b4a7fa39b/cf2: creating writer at 1675954460655 (+10 ms)Flushing 458ec544613af51e1d69337b4a7fa39b/cf2: appending metadata at 1675954460657 (+2 ms)Flushing 458ec544613af51e1d69337b4a7fa39b/cf2: closing flushed file at 1675954460657Flushing 458ec544613af51e1d69337b4a7fa39b/cf1: reopening flushed file at 1675954460667 (+10 ms)Flushing 458ec544613af51e1d69337b4a7fa39b/cf2: reopening flushed file at 1675954460672 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 458ec544613af51e1d69337b4a7fa39b in 35ms, sequenceid=82, compaction requested=true at 1675954460676 (+4 ms)Writing region close event to WAL at 1675954460679 (+3 ms)Running coprocessor post-close hooks at 1675954460682 (+3 ms)Closed at 1675954460682 2023-02-09 14:54:20,685 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:20,685 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=458ec544613af51e1d69337b4a7fa39b, regionState=CLOSED 2023-02-09 14:54:20,685 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675954460685"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954460685"}]},"ts":"1675954460685"} 2023-02-09 14:54:20,687 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:20,687 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:20,687 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:20,687 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:20,736 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:20,736 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:20,736 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:20,736 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:20,736 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:54:20,736 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:20,736 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:20,736 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:20,736 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:20,737 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:20,738 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 14:54:20,738 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-09 14:54:20,738 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-09 14:54:20,738 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:20,739 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:20,739 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:20,739 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:20,739 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:20,739 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:20,740 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:20,740 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:20,740 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:20,740 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:20,740 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:20,741 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:20,741 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:54:20,741 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:20,741 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN 2023-02-09 14:54:20,741 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:20,741 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:20,742 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:20,742 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:20,742 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:20,742 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:20,742 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN 2023-02-09 14:54:20,742 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:20,742 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 14:54:20,743 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:20,743 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:20,743 INFO [Listener at localhost/40823] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b to restore RIT 2023-02-09 14:54:20,744 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954490744 2023-02-09 14:54:20,744 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:20,745 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:20,747 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:20,747 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=458ec544613af51e1d69337b4a7fa39b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:20,747 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:20,747 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954520747 2023-02-09 14:54:20,747 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954580747 2023-02-09 14:54:20,747 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 14:54:20,747 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=458ec544613af51e1d69337b4a7fa39b, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:20,747 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675954460747"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954460747"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954460747"}]},"ts":"1675954460747"} 2023-02-09 14:54:20,749 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:54:20,751 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-09 14:54:20,751 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 in 262 msec 2023-02-09 14:54:20,751 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:20,751 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN 2023-02-09 14:54:20,751 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:20,752 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:20,997 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:20,997 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:20,998 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:20,998 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:20,999 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:54:20,999 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:20,999 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:20,999 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:20,999 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:21,000 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:21,001 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 14:54:21,001 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-09 14:54:21,001 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 14:54:21,002 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:21,002 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:21,002 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:21,002 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:21,003 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:21,003 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:21,003 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:21,003 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:21,003 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:21,004 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:21,004 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:21,004 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:21,004 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:54:21,005 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:21,005 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN 2023-02-09 14:54:21,005 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:21,005 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:21,005 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:21,005 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:21,005 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:21,005 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:21,005 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:21,006 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-09 14:54:21,006 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:21,007 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:21,007 INFO [Listener at localhost/40823] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b to restore RIT 2023-02-09 14:54:21,010 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954491010 2023-02-09 14:54:21,011 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:21,011 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN 2023-02-09 14:54:21,014 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:21,014 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=458ec544613af51e1d69337b4a7fa39b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:21,014 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:21,015 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954521015 2023-02-09 14:54:21,015 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954581015 2023-02-09 14:54:21,015 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 14:54:21,015 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN 2023-02-09 14:54:21,017 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:54:21,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-09 14:54:21,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN in 798 msec 2023-02-09 14:54:21,023 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:21,023 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:21,023 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:21,023 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:21,265 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:21,265 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:21,265 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:21,265 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:21,265 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:54:21,265 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:21,266 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:21,266 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:21,266 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:21,266 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:21,267 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 14:54:21,267 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 14:54:21,267 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 14:54:21,268 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:21,268 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:21,268 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:21,268 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:21,269 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:21,269 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:21,269 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:21,269 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:21,269 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:21,270 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:21,270 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:21,270 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:21,270 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:54:21,271 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:21,271 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN 2023-02-09 14:54:21,271 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:21,271 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:21,271 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:21,271 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:21,271 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 14:54:21,272 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:21,272 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:21,273 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954491273 2023-02-09 14:54:21,274 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:21,274 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:21,277 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:21,277 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=458ec544613af51e1d69337b4a7fa39b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:21,277 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:21,277 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954521277 2023-02-09 14:54:21,277 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954581277 2023-02-09 14:54:21,277 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 14:54:21,277 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:21,277 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:21,277 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:21,282 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:54:21,291 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=458ec544613af51e1d69337b4a7fa39b, threads=14 2023-02-09 14:54:21,291 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/1bedbf1b41144837aee62f61a6d624d0 for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,292 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/3eeee066fc07454b89a5897742cca0db for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,292 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/b5c988f26128498a84dd733de608ff49 for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,292 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/b6dfbc2187574a54ad15bb8661b211f4 for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,292 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/cf86fb5842b54944b820c99c213e667b for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,296 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/f1b8c01795c5490683a21355e270af0a for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,296 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/ff6b579b48cb4d36b866bf386f921a7f for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,296 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/04db8ad2f1be4c56b4710dcb46341919 for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,296 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/0b22449ce5bb43f8a79295863be22e8e for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,296 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/4e549ea7cb5040f2a1f0f8141f806241 for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,297 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/92085b16bd114633a8a9c119de934ddd for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,297 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/999689ceaa644f9db0bff84d94a40860 for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,297 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/b857c7123fc04f1fb52a1b13bef9712a for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,297 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/b8935f1f3a244c18b4524f10dc75d9fa for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,468 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/b5c988f26128498a84dd733de608ff49 for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,471 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/f1b8c01795c5490683a21355e270af0a for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,473 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/b6dfbc2187574a54ad15bb8661b211f4 for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,485 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/cf86fb5842b54944b820c99c213e667b for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,485 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/b8935f1f3a244c18b4524f10dc75d9fa for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,489 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/1bedbf1b41144837aee62f61a6d624d0 for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,490 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/04db8ad2f1be4c56b4710dcb46341919 for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,513 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/92085b16bd114633a8a9c119de934ddd for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,516 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/4e549ea7cb5040f2a1f0f8141f806241 for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,517 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/3eeee066fc07454b89a5897742cca0db for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,517 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/0b22449ce5bb43f8a79295863be22e8e for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,517 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/ff6b579b48cb4d36b866bf386f921a7f for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,519 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/b857c7123fc04f1fb52a1b13bef9712a for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,899 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/999689ceaa644f9db0bff84d94a40860 for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:21,899 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 458ec544613af51e1d69337b4a7fa39b Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 14:54:21,944 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:21,944 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:21,944 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:21,944 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:22,026 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 14:54:22,027 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,028 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:22,028 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:22,028 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:22,029 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:54:22,029 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:22,029 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:22,029 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:22,029 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:22,030 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:22,031 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-09 14:54:22,031 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 14:54:22,031 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 14:54:22,031 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:22,031 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:22,032 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:22,032 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:22,032 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:22,032 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:22,032 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:22,032 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:22,033 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:22,033 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:22,033 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:22,033 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:22,033 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:54:22,034 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,034 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN 2023-02-09 14:54:22,034 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:22,034 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:22,034 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,034 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,034 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,034 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:22,034 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:22,034 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 14:54:22,035 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:22,036 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:22,037 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954492037 2023-02-09 14:54:22,039 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,039 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:22,041 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:22,041 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=458ec544613af51e1d69337b4a7fa39b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:22,041 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:22,041 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954522041 2023-02-09 14:54:22,041 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954582041 2023-02-09 14:54:22,041 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 14:54:22,050 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=458ec544613af51e1d69337b4a7fa39b, threads=14 2023-02-09 14:54:22,051 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/1bedbf1b41144837aee62f61a6d624d0 for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,051 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/3eeee066fc07454b89a5897742cca0db for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,052 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/b5c988f26128498a84dd733de608ff49 for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,052 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/b6dfbc2187574a54ad15bb8661b211f4 for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,053 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/cf86fb5842b54944b820c99c213e667b for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,053 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/f1b8c01795c5490683a21355e270af0a for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,054 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/ff6b579b48cb4d36b866bf386f921a7f for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,054 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/0b22449ce5bb43f8a79295863be22e8e for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,055 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/4e549ea7cb5040f2a1f0f8141f806241 for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,055 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/92085b16bd114633a8a9c119de934ddd for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,055 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/999689ceaa644f9db0bff84d94a40860 for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,055 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/b857c7123fc04f1fb52a1b13bef9712a for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,055 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/b8935f1f3a244c18b4524f10dc75d9fa for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,057 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/04db8ad2f1be4c56b4710dcb46341919 for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,181 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/b5c988f26128498a84dd733de608ff49 for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,196 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/ff6b579b48cb4d36b866bf386f921a7f for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,199 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/1bedbf1b41144837aee62f61a6d624d0 for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,208 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/3eeee066fc07454b89a5897742cca0db for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,209 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/b857c7123fc04f1fb52a1b13bef9712a for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,209 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/04db8ad2f1be4c56b4710dcb46341919 for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,213 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/b6dfbc2187574a54ad15bb8661b211f4 for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,216 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/cf86fb5842b54944b820c99c213e667b for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,226 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/0b22449ce5bb43f8a79295863be22e8e for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,230 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/f1b8c01795c5490683a21355e270af0a for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,232 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/4e549ea7cb5040f2a1f0f8141f806241 for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,234 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/b8935f1f3a244c18b4524f10dc75d9fa for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,234 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/999689ceaa644f9db0bff84d94a40860 for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,237 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/92085b16bd114633a8a9c119de934ddd for region: 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:22,237 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 458ec544613af51e1d69337b4a7fa39b Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 14:54:22,273 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:54:22,279 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/485d653fadb7b7df523ff9ae46d1b78c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 14:54:22,280 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/d4498432e15515d5f43c5d2932f1cffa/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 14:54:22,280 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:22,280 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,280 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:22,281 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:22,291 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,292 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:22,292 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:22,292 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:22,293 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:54:22,293 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:22,293 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:22,293 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:22,293 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:22,294 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:22,295 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-09 14:54:22,295 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 14:54:22,295 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 14:54:22,295 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:22,295 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:22,295 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:22,296 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:22,296 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:22,296 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:22,296 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:22,296 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:22,297 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:22,297 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:22,297 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:22,297 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:22,297 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:54:22,298 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,298 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN 2023-02-09 14:54:22,298 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:22,301 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:22,301 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,301 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,301 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,301 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:22,301 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:22,301 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-09 14:54:22,302 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:22,302 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:22,303 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954492303 2023-02-09 14:54:22,303 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:22,304 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,307 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:22,307 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=458ec544613af51e1d69337b4a7fa39b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:22,307 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:22,307 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954522307 2023-02-09 14:54:22,307 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954582307 2023-02-09 14:54:22,307 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 14:54:22,310 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:54:22,315 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:22,315 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,315 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:22,315 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:22,558 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,558 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:22,558 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:22,558 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:22,558 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:54:22,559 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:22,559 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:22,559 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:22,559 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:22,560 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:22,561 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-09 14:54:22,561 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 14:54:22,561 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 14:54:22,561 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:22,562 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:22,562 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:22,562 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:22,563 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:22,563 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:22,563 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:22,563 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:22,564 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:22,564 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:22,564 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:22,565 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:22,565 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:54:22,566 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,566 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN 2023-02-09 14:54:22,566 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:22,566 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:22,566 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,566 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,566 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,567 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:22,567 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:22,567 DEBUG [Listener at localhost/40823] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,567 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-09 14:54:22,568 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:22,568 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:22,569 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954492569 2023-02-09 14:54:22,569 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:22,571 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,573 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:22,573 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=458ec544613af51e1d69337b4a7fa39b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:22,574 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:22,574 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954522574 2023-02-09 14:54:22,574 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954582574 2023-02-09 14:54:22,574 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-09 14:54:22,574 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:54:22,586 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:22,586 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 14:54:22,586 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:22,587 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:22,825 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,825 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:22,825 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:22,825 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:22,825 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:54:22,826 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:22,826 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:22,826 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:22,826 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:22,827 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:22,828 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-09 14:54:22,828 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 14:54:22,828 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 14:54:22,829 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:22,829 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:22,830 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:22,830 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:22,830 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:22,830 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:22,830 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:22,831 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:22,831 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:22,831 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:22,831 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:22,831 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:22,832 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:54:22,833 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,833 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN 2023-02-09 14:54:22,833 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:22,833 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:22,833 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,833 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,833 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,833 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:22,834 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:22,834 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 14:54:22,834 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:22,835 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:22,835 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954492835 2023-02-09 14:54:22,835 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:22,839 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:22,841 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:22,841 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=458ec544613af51e1d69337b4a7fa39b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:22,841 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:22,841 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954522841 2023-02-09 14:54:22,841 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954582841 2023-02-09 14:54:22,841 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 6 msec 2023-02-09 14:54:22,841 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:54:22,844 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:22,844 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 14:54:22,844 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:22,844 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:23,092 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:23,093 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:23,093 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:23,093 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:23,093 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:54:23,094 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:23,094 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:23,094 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:23,094 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:23,095 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:23,095 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-09 14:54:23,096 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 14:54:23,096 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 14:54:23,096 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:23,096 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:23,096 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:23,096 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:23,097 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:23,097 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:23,097 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:23,097 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:23,097 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:23,098 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:23,098 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:23,098 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:23,098 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:54:23,099 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:23,099 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN 2023-02-09 14:54:23,099 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:23,099 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:23,099 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:23,099 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:23,099 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:23,099 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:23,099 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:23,100 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 14:54:23,100 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:23,101 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:23,101 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954493101 2023-02-09 14:54:23,102 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:23,103 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:23,104 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:23,105 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=458ec544613af51e1d69337b4a7fa39b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:23,105 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:23,105 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954523105 2023-02-09 14:54:23,105 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954583105 2023-02-09 14:54:23,105 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 14:54:23,105 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:54:23,107 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:23,107 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 14:54:23,107 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:23,107 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:23,355 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:23,355 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:23,355 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:23,355 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:23,356 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:54:23,357 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:23,357 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:23,357 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:23,357 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:23,357 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:23,358 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-09 14:54:23,358 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 14:54:23,358 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 14:54:23,359 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:23,359 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:23,359 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:23,359 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:23,359 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:23,360 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:23,360 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:23,360 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:23,360 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:23,360 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:23,360 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:23,361 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:23,361 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:54:23,361 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:23,361 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN 2023-02-09 14:54:23,361 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:23,361 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:23,362 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:23,362 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:23,362 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:23,362 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:23,362 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:23,362 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 14:54:23,362 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:23,363 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:23,363 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954493363 2023-02-09 14:54:23,365 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:23,365 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:23,367 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:23,367 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=458ec544613af51e1d69337b4a7fa39b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:23,367 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:23,367 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954523367 2023-02-09 14:54:23,368 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954583368 2023-02-09 14:54:23,368 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 14:54:23,368 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:54:23,371 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN] 2023-02-09 14:54:23,371 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:23,372 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 14:54:23,372 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:23,372 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:23,618 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:23,618 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:23,618 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:23,618 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:23,618 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:54:23,618 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:23,618 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:23,619 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:23,619 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:23,619 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:23,620 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-09 14:54:23,620 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 14:54:23,620 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 14:54:23,620 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-09 14:54:23,620 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:23,621 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:23,621 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:23,621 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:23,622 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:23,622 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:23,622 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:23,622 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:23,622 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:23,623 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:23,623 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:23,623 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:23,623 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:54:23,625 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:23,625 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN 2023-02-09 14:54:23,625 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:23,625 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN 2023-02-09 14:54:23,625 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:23,625 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:23,625 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:23,625 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:23,625 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:23,625 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:23,626 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:23,626 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 14:54:23,626 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:23,627 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:23,627 INFO [Listener at localhost/40823] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b to restore RIT 2023-02-09 14:54:23,627 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954493627 2023-02-09 14:54:23,627 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:23,629 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:23,630 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN 2023-02-09 14:54:23,632 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:23,632 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=458ec544613af51e1d69337b4a7fa39b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:23,632 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:23,632 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954523632 2023-02-09 14:54:23,633 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954583632 2023-02-09 14:54:23,633 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-09 14:54:23,633 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:54:23,638 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:23,638 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 14:54:23,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:23,638 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:23,883 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:23,883 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:23,883 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:23,883 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:23,884 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:54:23,884 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:23,884 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:23,884 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:23,884 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:23,885 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:23,886 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 14:54:23,886 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-09 14:54:23,886 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:23,886 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:23,887 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:23,887 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:23,887 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:23,887 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:23,887 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:23,887 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:23,888 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:23,888 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:23,888 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:23,888 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:23,888 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:54:23,889 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:23,889 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN 2023-02-09 14:54:23,889 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:23,889 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:23,889 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:23,889 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:23,889 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:23,889 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 14:54:23,890 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:23,890 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:23,890 INFO [Listener at localhost/40823] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b to restore RIT 2023-02-09 14:54:23,891 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954493891 2023-02-09 14:54:23,892 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:23,893 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:23,894 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN 2023-02-09 14:54:23,895 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:23,896 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=458ec544613af51e1d69337b4a7fa39b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:23,896 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:23,896 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954523896 2023-02-09 14:54:23,896 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954583896 2023-02-09 14:54:23,896 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 14:54:23,896 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:54:23,898 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:23,898 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 14:54:23,898 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:23,898 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:24,146 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:24,146 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:24,146 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:24,146 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:24,149 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:54:24,149 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:24,149 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:24,149 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:24,149 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:24,150 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:24,151 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-09 14:54:24,151 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-09 14:54:24,151 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:24,151 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:24,151 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:24,151 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:24,152 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:24,152 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:24,152 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:24,152 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:24,153 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:24,153 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:24,153 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:24,153 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:24,153 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:54:24,154 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:24,154 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN 2023-02-09 14:54:24,154 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:24,154 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:24,154 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:24,154 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:24,154 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:24,154 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 14:54:24,155 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:24,155 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:24,155 INFO [Listener at localhost/40823] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b to restore RIT 2023-02-09 14:54:24,156 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954494156 2023-02-09 14:54:24,157 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:24,158 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:24,159 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN 2023-02-09 14:54:24,161 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:24,161 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=458ec544613af51e1d69337b4a7fa39b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:24,161 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:24,161 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954524161 2023-02-09 14:54:24,161 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954584161 2023-02-09 14:54:24,161 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 14:54:24,161 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:54:24,164 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:24,164 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-09 14:54:24,164 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:24,164 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:24,411 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:24,412 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:24,412 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:24,412 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:24,412 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:54:24,412 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:24,412 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:24,412 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:24,412 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:24,413 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:24,414 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-09 14:54:24,414 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-09 14:54:24,414 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:24,414 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:24,414 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:24,415 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:24,415 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:24,415 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:24,415 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:24,415 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:24,416 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:24,416 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:24,416 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:24,416 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:24,416 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:54:24,417 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:24,417 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN 2023-02-09 14:54:24,417 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:24,417 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:24,417 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:24,417 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:24,417 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:54:24,417 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-09 14:54:24,418 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:24,418 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:24,418 INFO [Listener at localhost/40823] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b to restore RIT 2023-02-09 14:54:24,419 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954494419 2023-02-09 14:54:24,419 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:24,421 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:24,423 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN 2023-02-09 14:54:24,425 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:24,425 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=458ec544613af51e1d69337b4a7fa39b, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:24,425 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:24,425 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954524425 2023-02-09 14:54:24,425 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954584425 2023-02-09 14:54:24,425 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 6 msec 2023-02-09 14:54:24,426 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:54:24,428 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa exec-time=4.9970 sec 2023-02-09 14:54:24,428 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN 2023-02-09 14:54:24,429 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,41077,1675954443212; forceNewPlan=false, retain=true 2023-02-09 14:54:24,579 INFO [jenkins-hbase4:43829] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 14:54:24,580 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=458ec544613af51e1d69337b4a7fa39b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:24,580 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675954464580"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954464580"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954464580"}]},"ts":"1675954464580"} 2023-02-09 14:54:24,582 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212}] 2023-02-09 14:54:24,582 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:54:24,582 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN 2023-02-09 14:54:24,582 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:54:24,582 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:54:24,676 WARN [Listener at localhost/40823] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 14:54:24,676 WARN [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-02-09 14:54:24,676 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:54:24,676 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:54:24,676 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:54:24,676 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:54:24,676 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:54:24,676 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:54:24,676 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:54:24,676 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:54:24,677 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:54:24,678 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-02-09 14:54:24,678 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:54:24,679 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:54:24,679 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:54:24,679 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:54:24,679 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:24,679 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:54:24,679 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:24,680 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:54:24,680 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:24,680 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:54:24,680 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:24,680 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:54:24,680 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:54:24,681 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:54:24,681 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN 2023-02-09 14:54:24,681 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:54:24,681 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN 2023-02-09 14:54:24,681 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-02-09 14:54:24,682 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:54:24,682 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:54:24,682 INFO [Listener at localhost/40823] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b to restore RIT 2023-02-09 14:54:24,683 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954494683 2023-02-09 14:54:24,684 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:54:24,684 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN 2023-02-09 14:54:24,686 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:54:24,686 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=458ec544613af51e1d69337b4a7fa39b, regionState=OPENING, lastHost=jenkins-hbase4.apache.org,41077,1675954443212, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212, openSeqNum=2 2023-02-09 14:54:24,686 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:54:24,686 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954524686 2023-02-09 14:54:24,686 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954584686 2023-02-09 14:54:24,686 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 14:54:24,687 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=458ec544613af51e1d69337b4a7fa39b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:24,687 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675954464687"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954464687"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954464687"}]},"ts":"1675954464687"} 2023-02-09 14:54:24,688 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212}] 2023-02-09 14:54:24,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:24,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 458ec544613af51e1d69337b4a7fa39b, NAME => 'testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:54:24,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:54:24,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:24,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:24,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:24,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:24,846 INFO [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:24,847 DEBUG [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1 2023-02-09 14:54:24,847 DEBUG [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1 2023-02-09 14:54:24,847 INFO [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 458ec544613af51e1d69337b4a7fa39b columnFamilyName cf1 2023-02-09 14:54:24,853 DEBUG [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/04db8ad2f1be4c56b4710dcb46341919 2023-02-09 14:54:24,857 DEBUG [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/0b22449ce5bb43f8a79295863be22e8e 2023-02-09 14:54:24,859 DEBUG [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/4e549ea7cb5040f2a1f0f8141f806241 2023-02-09 14:54:24,863 DEBUG [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/92085b16bd114633a8a9c119de934ddd 2023-02-09 14:54:24,866 DEBUG [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/999689ceaa644f9db0bff84d94a40860 2023-02-09 14:54:24,869 DEBUG [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/b857c7123fc04f1fb52a1b13bef9712a 2023-02-09 14:54:24,872 DEBUG [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/b8935f1f3a244c18b4524f10dc75d9fa 2023-02-09 14:54:24,872 INFO [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] regionserver.HStore(310): Store=458ec544613af51e1d69337b4a7fa39b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:24,872 INFO [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:24,873 DEBUG [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2 2023-02-09 14:54:24,873 DEBUG [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2 2023-02-09 14:54:24,873 INFO [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 458ec544613af51e1d69337b4a7fa39b columnFamilyName cf2 2023-02-09 14:54:24,880 DEBUG [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/1bedbf1b41144837aee62f61a6d624d0 2023-02-09 14:54:24,883 DEBUG [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/3eeee066fc07454b89a5897742cca0db 2023-02-09 14:54:24,886 DEBUG [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/b5c988f26128498a84dd733de608ff49 2023-02-09 14:54:24,889 DEBUG [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/b6dfbc2187574a54ad15bb8661b211f4 2023-02-09 14:54:24,892 DEBUG [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/cf86fb5842b54944b820c99c213e667b 2023-02-09 14:54:24,896 DEBUG [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/f1b8c01795c5490683a21355e270af0a 2023-02-09 14:54:24,899 DEBUG [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/ff6b579b48cb4d36b866bf386f921a7f 2023-02-09 14:54:24,899 INFO [StoreOpener-458ec544613af51e1d69337b4a7fa39b-1] regionserver.HStore(310): Store=458ec544613af51e1d69337b4a7fa39b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:24,900 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:24,901 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:24,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.splits 2023-02-09 14:54:24,904 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:54:24,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:24,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 458ec544613af51e1d69337b4a7fa39b; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59320798, jitterRate=-0.11605122685432434}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:54:24,906 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 458ec544613af51e1d69337b4a7fa39b: Running coprocessor pre-open hook at 1675954464845Writing region info on filesystem at 1675954464845Initializing all the Stores at 1675954464845Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954464845Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954464846 (+1 ms)Cleaning up temporary data from old regions at 1675954464902 (+56 ms)Cleaning up detritus from prior splits at 1675954464902Running coprocessor post-open hooks at 1675954464906 (+4 ms)Region opened successfully at 1675954464906 2023-02-09 14:54:24,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b., pid=48, masterSystemTime=1675954464840 2023-02-09 14:54:24,907 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:24,908 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 14:54:24,908 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 14:54:24,908 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 14:54:24,909 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-09 14:54:24,909 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-09 14:54:24,909 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStore(1912): 458ec544613af51e1d69337b4a7fa39b/cf1 is initiating minor compaction (all files) 2023-02-09 14:54:24,909 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStore(1912): 458ec544613af51e1d69337b4a7fa39b/cf2 is initiating minor compaction (all files) 2023-02-09 14:54:24,909 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 458ec544613af51e1d69337b4a7fa39b/cf1 in testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:24,909 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 458ec544613af51e1d69337b4a7fa39b/cf2 in testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:24,909 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/b8935f1f3a244c18b4524f10dc75d9fa, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/b857c7123fc04f1fb52a1b13bef9712a, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/0b22449ce5bb43f8a79295863be22e8e, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/04db8ad2f1be4c56b4710dcb46341919, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/92085b16bd114633a8a9c119de934ddd, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/999689ceaa644f9db0bff84d94a40860, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/4e549ea7cb5040f2a1f0f8141f806241] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp, totalSize=34.9 K 2023-02-09 14:54:24,909 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/cf86fb5842b54944b820c99c213e667b, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/ff6b579b48cb4d36b866bf386f921a7f, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/b5c988f26128498a84dd733de608ff49, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/1bedbf1b41144837aee62f61a6d624d0, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/3eeee066fc07454b89a5897742cca0db, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/b6dfbc2187574a54ad15bb8661b211f4, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/f1b8c01795c5490683a21355e270af0a] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp, totalSize=34.9 K 2023-02-09 14:54:24,911 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:24,911 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting b8935f1f3a244c18b4524f10dc75d9fa, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675954458025 2023-02-09 14:54:24,911 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting cf86fb5842b54944b820c99c213e667b, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675954458025 2023-02-09 14:54:24,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:24,911 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting b857c7123fc04f1fb52a1b13bef9712a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675954458213 2023-02-09 14:54:24,911 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting ff6b579b48cb4d36b866bf386f921a7f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675954458213 2023-02-09 14:54:24,911 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=458ec544613af51e1d69337b4a7fa39b, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:24,911 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675954464911"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954464911"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954464911"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954464911"}]},"ts":"1675954464911"} 2023-02-09 14:54:24,911 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 0b22449ce5bb43f8a79295863be22e8e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675954458405 2023-02-09 14:54:24,912 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting b5c988f26128498a84dd733de608ff49, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675954458405 2023-02-09 14:54:24,912 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 04db8ad2f1be4c56b4710dcb46341919, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675954458611 2023-02-09 14:54:24,912 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting 1bedbf1b41144837aee62f61a6d624d0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675954458611 2023-02-09 14:54:24,912 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 92085b16bd114633a8a9c119de934ddd, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675954458797 2023-02-09 14:54:24,912 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting 3eeee066fc07454b89a5897742cca0db, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675954458797 2023-02-09 14:54:24,913 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 999689ceaa644f9db0bff84d94a40860, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675954458987 2023-02-09 14:54:24,913 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting b6dfbc2187574a54ad15bb8661b211f4, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675954458987 2023-02-09 14:54:24,913 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 4e549ea7cb5040f2a1f0f8141f806241, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675954459168 2023-02-09 14:54:24,913 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting f1b8c01795c5490683a21355e270af0a, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675954459168 2023-02-09 14:54:24,917 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-02-09 14:54:24,917 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 in 225 msec 2023-02-09 14:54:24,919 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN in 1.5470 sec 2023-02-09 14:54:24,943 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] throttle.PressureAwareThroughputController(145): 458ec544613af51e1d69337b4a7fa39b#cf1#compaction#50 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:54:24,944 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] throttle.PressureAwareThroughputController(145): 458ec544613af51e1d69337b4a7fa39b#cf2#compaction#51 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:54:24,968 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf1/9ab6a42cbaf84bd994452869a1ba9900 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/9ab6a42cbaf84bd994452869a1ba9900 2023-02-09 14:54:24,968 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/.tmp/cf2/6b49fa97e623428697f69c5799bedff5 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/6b49fa97e623428697f69c5799bedff5 2023-02-09 14:54:24,973 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 458ec544613af51e1d69337b4a7fa39b/cf2 of 458ec544613af51e1d69337b4a7fa39b into 6b49fa97e623428697f69c5799bedff5(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:54:24,973 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 458ec544613af51e1d69337b4a7fa39b/cf1 of 458ec544613af51e1d69337b4a7fa39b into 9ab6a42cbaf84bd994452869a1ba9900(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:54:24,974 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 458ec544613af51e1d69337b4a7fa39b: 2023-02-09 14:54:24,974 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 458ec544613af51e1d69337b4a7fa39b: 2023-02-09 14:54:24,974 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b., storeName=458ec544613af51e1d69337b4a7fa39b/cf2, priority=9, startTime=1675954464908; duration=0sec 2023-02-09 14:54:24,974 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b., storeName=458ec544613af51e1d69337b4a7fa39b/cf1, priority=9, startTime=1675954464907; duration=0sec 2023-02-09 14:54:24,975 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:24,975 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:25,263 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-02-09 14:54:25,265 DEBUG [Listener at localhost/40823] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-02-09 14:54:25,267 INFO [Listener at localhost/40823] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-09 14:54:25,269 INFO [Listener at localhost/40823] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-02-09 14:54:25,270 INFO [Listener at localhost/40823] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-09 14:54:25,270 INFO [Listener at localhost/40823] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-02-09 14:54:25,286 WARN [Listener at localhost/40823] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 14:54:25,286 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:25,286 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:25,287 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:25,288 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:25,288 INFO [Listener at localhost/40823] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-02-09 14:54:25,288 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackAndDoubleExecution 2023-02-09 14:54:25,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:54:25,291 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:25,291 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:25,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-09 14:54:25,292 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954465292"}]},"ts":"1675954465292"} 2023-02-09 14:54:25,293 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-09 14:54:25,294 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-02-09 14:54:25,295 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN}] 2023-02-09 14:54:25,295 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN 2023-02-09 14:54:25,296 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=458ec544613af51e1d69337b4a7fa39b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:25,296 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675954465296"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954465296"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954465296"}]},"ts":"1675954465296"} 2023-02-09 14:54:25,297 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212}] 2023-02-09 14:54:25,392 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:25,392 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:25,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-09 14:54:25,449 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:25,449 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 458ec544613af51e1d69337b4a7fa39b, disabling compactions & flushes 2023-02-09 14:54:25,449 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:25,449 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:25,449 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. after waiting 0 ms 2023-02-09 14:54:25,449 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:25,450 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/b8935f1f3a244c18b4524f10dc75d9fa, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/b857c7123fc04f1fb52a1b13bef9712a, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/0b22449ce5bb43f8a79295863be22e8e, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/04db8ad2f1be4c56b4710dcb46341919, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/92085b16bd114633a8a9c119de934ddd, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/999689ceaa644f9db0bff84d94a40860, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/4e549ea7cb5040f2a1f0f8141f806241] to archive 2023-02-09 14:54:25,451 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:25,453 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/b8935f1f3a244c18b4524f10dc75d9fa to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/b8935f1f3a244c18b4524f10dc75d9fa 2023-02-09 14:54:25,454 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/b857c7123fc04f1fb52a1b13bef9712a to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/b857c7123fc04f1fb52a1b13bef9712a 2023-02-09 14:54:25,455 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/0b22449ce5bb43f8a79295863be22e8e to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/0b22449ce5bb43f8a79295863be22e8e 2023-02-09 14:54:25,456 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/04db8ad2f1be4c56b4710dcb46341919 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/04db8ad2f1be4c56b4710dcb46341919 2023-02-09 14:54:25,457 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/92085b16bd114633a8a9c119de934ddd to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/92085b16bd114633a8a9c119de934ddd 2023-02-09 14:54:25,459 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/999689ceaa644f9db0bff84d94a40860 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/999689ceaa644f9db0bff84d94a40860 2023-02-09 14:54:25,460 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/4e549ea7cb5040f2a1f0f8141f806241 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/4e549ea7cb5040f2a1f0f8141f806241 2023-02-09 14:54:25,464 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/cf86fb5842b54944b820c99c213e667b, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/ff6b579b48cb4d36b866bf386f921a7f, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/b5c988f26128498a84dd733de608ff49, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/1bedbf1b41144837aee62f61a6d624d0, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/3eeee066fc07454b89a5897742cca0db, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/b6dfbc2187574a54ad15bb8661b211f4, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/f1b8c01795c5490683a21355e270af0a] to archive 2023-02-09 14:54:25,464 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:25,466 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/cf86fb5842b54944b820c99c213e667b to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/cf86fb5842b54944b820c99c213e667b 2023-02-09 14:54:25,467 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/ff6b579b48cb4d36b866bf386f921a7f to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/ff6b579b48cb4d36b866bf386f921a7f 2023-02-09 14:54:25,468 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/b5c988f26128498a84dd733de608ff49 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/b5c988f26128498a84dd733de608ff49 2023-02-09 14:54:25,469 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/1bedbf1b41144837aee62f61a6d624d0 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/1bedbf1b41144837aee62f61a6d624d0 2023-02-09 14:54:25,470 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/3eeee066fc07454b89a5897742cca0db to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/3eeee066fc07454b89a5897742cca0db 2023-02-09 14:54:25,471 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/b6dfbc2187574a54ad15bb8661b211f4 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/b6dfbc2187574a54ad15bb8661b211f4 2023-02-09 14:54:25,472 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/f1b8c01795c5490683a21355e270af0a to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/f1b8c01795c5490683a21355e270af0a 2023-02-09 14:54:25,475 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 14:54:25,476 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:54:25,476 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b. 2023-02-09 14:54:25,476 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 458ec544613af51e1d69337b4a7fa39b: Waiting for close lock at 1675954465449Running coprocessor pre-close hooks at 1675954465449Disabling compacts and flushes for region at 1675954465449Disabling writes for close at 1675954465449Writing region close event to WAL at 1675954465473 (+24 ms)Running coprocessor post-close hooks at 1675954465476 (+3 ms)Closed at 1675954465476 2023-02-09 14:54:25,477 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:25,478 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=458ec544613af51e1d69337b4a7fa39b, regionState=CLOSED 2023-02-09 14:54:25,478 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675954465478"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954465478"}]},"ts":"1675954465478"} 2023-02-09 14:54:25,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-09 14:54:25,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure 458ec544613af51e1d69337b4a7fa39b, server=jenkins-hbase4.apache.org,41077,1675954443212 in 182 msec 2023-02-09 14:54:25,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-09 14:54:25,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, UNASSIGN in 186 msec 2023-02-09 14:54:25,484 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954465483"}]},"ts":"1675954465483"} 2023-02-09 14:54:25,485 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-09 14:54:25,486 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-02-09 14:54:25,488 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 199 msec 2023-02-09 14:54:25,593 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:25,593 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:25,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-09 14:54:25,595 INFO [Listener at localhost/40823] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-02-09 14:54:25,595 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:25,595 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:25,596 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackAndDoubleExecution 2023-02-09 14:54:25,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:54:25,598 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:54:25,598 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:25,598 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:25,599 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:54:25,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-09 14:54:25,602 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:25,604 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/recovered.edits] 2023-02-09 14:54:25,607 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/9ab6a42cbaf84bd994452869a1ba9900 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf1/9ab6a42cbaf84bd994452869a1ba9900 2023-02-09 14:54:25,609 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/6b49fa97e623428697f69c5799bedff5 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/cf2/6b49fa97e623428697f69c5799bedff5 2023-02-09 14:54:25,611 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/recovered.edits/90.seqid to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b/recovered.edits/90.seqid 2023-02-09 14:54:25,612 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRollbackAndDoubleExecution/458ec544613af51e1d69337b4a7fa39b 2023-02-09 14:54:25,612 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-09 14:54:25,615 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:54:25,617 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-02-09 14:54:25,619 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-02-09 14:54:25,619 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:54:25,619 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-02-09 14:54:25,620 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954465620"}]},"ts":"9223372036854775807"} 2023-02-09 14:54:25,621 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-09 14:54:25,621 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 458ec544613af51e1d69337b4a7fa39b, NAME => 'testRollbackAndDoubleExecution,,1675954457520.458ec544613af51e1d69337b4a7fa39b.', STARTKEY => '', ENDKEY => ''}] 2023-02-09 14:54:25,621 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-02-09 14:54:25,621 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675954465621"}]},"ts":"9223372036854775807"} 2023-02-09 14:54:25,622 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-02-09 14:54:25,624 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:54:25,625 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 28 msec 2023-02-09 14:54:25,699 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:25,700 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:25,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-09 14:54:25,701 INFO [Listener at localhost/40823] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-02-09 14:54:25,715 INFO [Listener at localhost/40823] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=397 (was 401), OpenFileDescriptor=853 (was 855), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=237 (was 281), ProcessCount=168 (was 168), AvailableMemoryMB=3224 (was 3240) 2023-02-09 14:54:25,729 INFO [Listener at localhost/40823] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=397, OpenFileDescriptor=853, MaxFileDescriptor=60000, SystemLoadAverage=237, ProcessCount=168, AvailableMemoryMB=3223 2023-02-09 14:54:25,729 WARN [Listener at localhost/40823] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 14:54:25,729 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:25,729 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:25,733 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 14:54:25,733 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 14:54:25,734 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:54:25,736 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 14:54:25,736 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 14:54:25,738 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionNoStoreFile/7a3a2a2c1be8b68cf6b9c5576ca1d967 2023-02-09 14:54:25,739 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionNoStoreFile/7a3a2a2c1be8b68cf6b9c5576ca1d967 empty. 2023-02-09 14:54:25,739 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionNoStoreFile/7a3a2a2c1be8b68cf6b9c5576ca1d967 2023-02-09 14:54:25,739 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-09 14:54:25,755 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-02-09 14:54:25,756 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7671): creating {ENCODED => 7a3a2a2c1be8b68cf6b9c5576ca1d967, NAME => 'testSplitTableRegionNoStoreFile,,1675954465733.7a3a2a2c1be8b68cf6b9c5576ca1d967.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp 2023-02-09 14:54:25,774 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675954465733.7a3a2a2c1be8b68cf6b9c5576ca1d967.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:25,775 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1603): Closing 7a3a2a2c1be8b68cf6b9c5576ca1d967, disabling compactions & flushes 2023-02-09 14:54:25,775 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675954465733.7a3a2a2c1be8b68cf6b9c5576ca1d967. 2023-02-09 14:54:25,775 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675954465733.7a3a2a2c1be8b68cf6b9c5576ca1d967. 2023-02-09 14:54:25,775 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675954465733.7a3a2a2c1be8b68cf6b9c5576ca1d967. after waiting 0 ms 2023-02-09 14:54:25,775 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675954465733.7a3a2a2c1be8b68cf6b9c5576ca1d967. 2023-02-09 14:54:25,775 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675954465733.7a3a2a2c1be8b68cf6b9c5576ca1d967. 2023-02-09 14:54:25,775 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1557): Region close journal for 7a3a2a2c1be8b68cf6b9c5576ca1d967: Waiting for close lock at 1675954465775Disabling compacts and flushes for region at 1675954465775Disabling writes for close at 1675954465775Writing region close event to WAL at 1675954465775Closed at 1675954465775 2023-02-09 14:54:25,777 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 14:54:25,778 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675954465733.7a3a2a2c1be8b68cf6b9c5576ca1d967.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675954465778"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954465778"}]},"ts":"1675954465778"} 2023-02-09 14:54:25,780 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 14:54:25,780 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 14:54:25,780 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954465780"}]},"ts":"1675954465780"} 2023-02-09 14:54:25,782 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-02-09 14:54:25,784 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 14:54:25,785 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 14:54:25,785 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 14:54:25,785 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 14:54:25,785 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 14:54:25,786 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=7a3a2a2c1be8b68cf6b9c5576ca1d967, ASSIGN}] 2023-02-09 14:54:25,787 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=7a3a2a2c1be8b68cf6b9c5576ca1d967, ASSIGN 2023-02-09 14:54:25,788 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=7a3a2a2c1be8b68cf6b9c5576ca1d967, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33423,1675954443133; forceNewPlan=false, retain=false 2023-02-09 14:54:25,938 INFO [jenkins-hbase4:43829] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 14:54:25,939 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=7a3a2a2c1be8b68cf6b9c5576ca1d967, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:25,939 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675954465733.7a3a2a2c1be8b68cf6b9c5576ca1d967.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675954465939"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954465939"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954465939"}]},"ts":"1675954465939"} 2023-02-09 14:54:25,941 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure 7a3a2a2c1be8b68cf6b9c5576ca1d967, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:54:26,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1675954465733.7a3a2a2c1be8b68cf6b9c5576ca1d967. 2023-02-09 14:54:26,099 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7a3a2a2c1be8b68cf6b9c5576ca1d967, NAME => 'testSplitTableRegionNoStoreFile,,1675954465733.7a3a2a2c1be8b68cf6b9c5576ca1d967.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:54:26,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:54:26,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 7a3a2a2c1be8b68cf6b9c5576ca1d967 2023-02-09 14:54:26,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675954465733.7a3a2a2c1be8b68cf6b9c5576ca1d967.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:26,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 7a3a2a2c1be8b68cf6b9c5576ca1d967 2023-02-09 14:54:26,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 7a3a2a2c1be8b68cf6b9c5576ca1d967 2023-02-09 14:54:26,101 INFO [StoreOpener-7a3a2a2c1be8b68cf6b9c5576ca1d967-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7a3a2a2c1be8b68cf6b9c5576ca1d967 2023-02-09 14:54:26,102 DEBUG [StoreOpener-7a3a2a2c1be8b68cf6b9c5576ca1d967-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionNoStoreFile/7a3a2a2c1be8b68cf6b9c5576ca1d967/cf1 2023-02-09 14:54:26,102 DEBUG [StoreOpener-7a3a2a2c1be8b68cf6b9c5576ca1d967-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionNoStoreFile/7a3a2a2c1be8b68cf6b9c5576ca1d967/cf1 2023-02-09 14:54:26,103 INFO [StoreOpener-7a3a2a2c1be8b68cf6b9c5576ca1d967-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7a3a2a2c1be8b68cf6b9c5576ca1d967 columnFamilyName cf1 2023-02-09 14:54:26,104 INFO [StoreOpener-7a3a2a2c1be8b68cf6b9c5576ca1d967-1] regionserver.HStore(310): Store=7a3a2a2c1be8b68cf6b9c5576ca1d967/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:26,104 INFO [StoreOpener-7a3a2a2c1be8b68cf6b9c5576ca1d967-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7a3a2a2c1be8b68cf6b9c5576ca1d967 2023-02-09 14:54:26,105 DEBUG [StoreOpener-7a3a2a2c1be8b68cf6b9c5576ca1d967-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionNoStoreFile/7a3a2a2c1be8b68cf6b9c5576ca1d967/cf2 2023-02-09 14:54:26,105 DEBUG [StoreOpener-7a3a2a2c1be8b68cf6b9c5576ca1d967-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionNoStoreFile/7a3a2a2c1be8b68cf6b9c5576ca1d967/cf2 2023-02-09 14:54:26,106 INFO [StoreOpener-7a3a2a2c1be8b68cf6b9c5576ca1d967-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7a3a2a2c1be8b68cf6b9c5576ca1d967 columnFamilyName cf2 2023-02-09 14:54:26,106 INFO [StoreOpener-7a3a2a2c1be8b68cf6b9c5576ca1d967-1] regionserver.HStore(310): Store=7a3a2a2c1be8b68cf6b9c5576ca1d967/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:26,107 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionNoStoreFile/7a3a2a2c1be8b68cf6b9c5576ca1d967 2023-02-09 14:54:26,107 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionNoStoreFile/7a3a2a2c1be8b68cf6b9c5576ca1d967 2023-02-09 14:54:26,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:54:26,110 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 7a3a2a2c1be8b68cf6b9c5576ca1d967 2023-02-09 14:54:26,112 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionNoStoreFile/7a3a2a2c1be8b68cf6b9c5576ca1d967/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:54:26,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 7a3a2a2c1be8b68cf6b9c5576ca1d967; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62083143, jitterRate=-0.07488907873630524}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:54:26,112 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 7a3a2a2c1be8b68cf6b9c5576ca1d967: Running coprocessor pre-open hook at 1675954466100Writing region info on filesystem at 1675954466100Initializing all the Stores at 1675954466101 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954466101Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954466101Cleaning up temporary data from old regions at 1675954466108 (+7 ms)Cleaning up detritus from prior splits at 1675954466108Running coprocessor post-open hooks at 1675954466112 (+4 ms)Region opened successfully at 1675954466112 2023-02-09 14:54:26,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1675954465733.7a3a2a2c1be8b68cf6b9c5576ca1d967., pid=55, masterSystemTime=1675954466092 2023-02-09 14:54:26,115 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1675954465733.7a3a2a2c1be8b68cf6b9c5576ca1d967. 2023-02-09 14:54:26,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1675954465733.7a3a2a2c1be8b68cf6b9c5576ca1d967. 2023-02-09 14:54:26,115 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=7a3a2a2c1be8b68cf6b9c5576ca1d967, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:26,115 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1675954465733.7a3a2a2c1be8b68cf6b9c5576ca1d967.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675954466115"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954466115"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954466115"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954466115"}]},"ts":"1675954466115"} 2023-02-09 14:54:26,118 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-02-09 14:54:26,118 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure 7a3a2a2c1be8b68cf6b9c5576ca1d967, server=jenkins-hbase4.apache.org,33423,1675954443133 in 175 msec 2023-02-09 14:54:26,120 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-02-09 14:54:26,120 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=7a3a2a2c1be8b68cf6b9c5576ca1d967, ASSIGN in 333 msec 2023-02-09 14:54:26,121 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 14:54:26,121 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954466121"}]},"ts":"1675954466121"} 2023-02-09 14:54:26,122 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-02-09 14:54:26,124 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 14:54:26,125 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 391 msec 2023-02-09 14:54:26,235 DEBUG [Listener at localhost/40823] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:26,236 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7a3a2a2c1be8b68cf6b9c5576ca1d967, daughterA=915612316d6a09fe13e83a17e2d06051, daughterB=2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:54:26,238 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7a3a2a2c1be8b68cf6b9c5576ca1d967, daughterA=915612316d6a09fe13e83a17e2d06051, daughterB=2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:54:26,238 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7a3a2a2c1be8b68cf6b9c5576ca1d967, daughterA=915612316d6a09fe13e83a17e2d06051, daughterB=2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:54:26,238 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7a3a2a2c1be8b68cf6b9c5576ca1d967, daughterA=915612316d6a09fe13e83a17e2d06051, daughterB=2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:54:26,240 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=7a3a2a2c1be8b68cf6b9c5576ca1d967, UNASSIGN}] 2023-02-09 14:54:26,240 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=7a3a2a2c1be8b68cf6b9c5576ca1d967, UNASSIGN 2023-02-09 14:54:26,241 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=7a3a2a2c1be8b68cf6b9c5576ca1d967, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:26,241 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675954465733.7a3a2a2c1be8b68cf6b9c5576ca1d967.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675954466241"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954466241"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954466241"}]},"ts":"1675954466241"} 2023-02-09 14:54:26,243 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure 7a3a2a2c1be8b68cf6b9c5576ca1d967, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:54:26,395 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 7a3a2a2c1be8b68cf6b9c5576ca1d967 2023-02-09 14:54:26,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 7a3a2a2c1be8b68cf6b9c5576ca1d967, disabling compactions & flushes 2023-02-09 14:54:26,396 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675954465733.7a3a2a2c1be8b68cf6b9c5576ca1d967. 2023-02-09 14:54:26,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675954465733.7a3a2a2c1be8b68cf6b9c5576ca1d967. 2023-02-09 14:54:26,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675954465733.7a3a2a2c1be8b68cf6b9c5576ca1d967. after waiting 0 ms 2023-02-09 14:54:26,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675954465733.7a3a2a2c1be8b68cf6b9c5576ca1d967. 2023-02-09 14:54:26,400 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionNoStoreFile/7a3a2a2c1be8b68cf6b9c5576ca1d967/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-09 14:54:26,400 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:54:26,401 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675954465733.7a3a2a2c1be8b68cf6b9c5576ca1d967. 2023-02-09 14:54:26,401 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 7a3a2a2c1be8b68cf6b9c5576ca1d967: Waiting for close lock at 1675954466395Running coprocessor pre-close hooks at 1675954466395Disabling compacts and flushes for region at 1675954466395Disabling writes for close at 1675954466396 (+1 ms)Writing region close event to WAL at 1675954466396Running coprocessor post-close hooks at 1675954466400 (+4 ms)Closed at 1675954466401 (+1 ms) 2023-02-09 14:54:26,402 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 7a3a2a2c1be8b68cf6b9c5576ca1d967 2023-02-09 14:54:26,403 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=7a3a2a2c1be8b68cf6b9c5576ca1d967, regionState=CLOSED 2023-02-09 14:54:26,403 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675954465733.7a3a2a2c1be8b68cf6b9c5576ca1d967.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675954466403"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954466403"}]},"ts":"1675954466403"} 2023-02-09 14:54:26,406 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-02-09 14:54:26,406 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure 7a3a2a2c1be8b68cf6b9c5576ca1d967, server=jenkins-hbase4.apache.org,33423,1675954443133 in 161 msec 2023-02-09 14:54:26,408 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-09 14:54:26,408 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=7a3a2a2c1be8b68cf6b9c5576ca1d967, UNASSIGN in 166 msec 2023-02-09 14:54:26,837 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionNoStoreFile/915612316d6a09fe13e83a17e2d06051/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-09 14:54:26,839 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionNoStoreFile/2c62e1c084902d2aa62c5102c5ff76cd/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-09 14:54:26,840 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675954465733.7a3a2a2c1be8b68cf6b9c5576ca1d967.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675954466840"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1675954466840"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1675954466840"}]},"ts":"1675954466840"} 2023-02-09 14:54:26,840 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675954466236.915612316d6a09fe13e83a17e2d06051.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675954466840"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954466840"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954466840"}]},"ts":"1675954466840"} 2023-02-09 14:54:26,841 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675954466236.2c62e1c084902d2aa62c5102c5ff76cd.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675954466840"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954466840"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954466840"}]},"ts":"1675954466840"} 2023-02-09 14:54:26,843 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=915612316d6a09fe13e83a17e2d06051, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2c62e1c084902d2aa62c5102c5ff76cd, ASSIGN}] 2023-02-09 14:54:26,844 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2c62e1c084902d2aa62c5102c5ff76cd, ASSIGN 2023-02-09 14:54:26,845 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2c62e1c084902d2aa62c5102c5ff76cd, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33423,1675954443133; forceNewPlan=false, retain=false 2023-02-09 14:54:26,845 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=915612316d6a09fe13e83a17e2d06051, ASSIGN 2023-02-09 14:54:26,846 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=915612316d6a09fe13e83a17e2d06051, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33423,1675954443133; forceNewPlan=false, retain=false 2023-02-09 14:54:26,995 INFO [jenkins-hbase4:43829] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 14:54:26,997 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=2c62e1c084902d2aa62c5102c5ff76cd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:26,997 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675954466236.2c62e1c084902d2aa62c5102c5ff76cd.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675954466997"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954466997"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954466997"}]},"ts":"1675954466997"} 2023-02-09 14:54:26,999 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE; OpenRegionProcedure 2c62e1c084902d2aa62c5102c5ff76cd, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:54:27,001 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=915612316d6a09fe13e83a17e2d06051, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:27,001 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675954466236.915612316d6a09fe13e83a17e2d06051.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675954467001"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954467001"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954467001"}]},"ts":"1675954467001"} 2023-02-09 14:54:27,002 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=59, state=RUNNABLE; OpenRegionProcedure 915612316d6a09fe13e83a17e2d06051, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:54:27,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1675954466236.915612316d6a09fe13e83a17e2d06051. 2023-02-09 14:54:27,153 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 915612316d6a09fe13e83a17e2d06051, NAME => 'testSplitTableRegionNoStoreFile,,1675954466236.915612316d6a09fe13e83a17e2d06051.', STARTKEY => '', ENDKEY => '41'} 2023-02-09 14:54:27,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:54:27,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 915612316d6a09fe13e83a17e2d06051 2023-02-09 14:54:27,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1675954466236.915612316d6a09fe13e83a17e2d06051.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:27,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 915612316d6a09fe13e83a17e2d06051 2023-02-09 14:54:27,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 915612316d6a09fe13e83a17e2d06051 2023-02-09 14:54:27,155 INFO [StoreOpener-915612316d6a09fe13e83a17e2d06051-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 915612316d6a09fe13e83a17e2d06051 2023-02-09 14:54:27,156 DEBUG [StoreOpener-915612316d6a09fe13e83a17e2d06051-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionNoStoreFile/915612316d6a09fe13e83a17e2d06051/cf1 2023-02-09 14:54:27,156 DEBUG [StoreOpener-915612316d6a09fe13e83a17e2d06051-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionNoStoreFile/915612316d6a09fe13e83a17e2d06051/cf1 2023-02-09 14:54:27,156 INFO [StoreOpener-915612316d6a09fe13e83a17e2d06051-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 915612316d6a09fe13e83a17e2d06051 columnFamilyName cf1 2023-02-09 14:54:27,157 INFO [StoreOpener-915612316d6a09fe13e83a17e2d06051-1] regionserver.HStore(310): Store=915612316d6a09fe13e83a17e2d06051/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:27,157 INFO [StoreOpener-915612316d6a09fe13e83a17e2d06051-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 915612316d6a09fe13e83a17e2d06051 2023-02-09 14:54:27,158 DEBUG [StoreOpener-915612316d6a09fe13e83a17e2d06051-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionNoStoreFile/915612316d6a09fe13e83a17e2d06051/cf2 2023-02-09 14:54:27,158 DEBUG [StoreOpener-915612316d6a09fe13e83a17e2d06051-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionNoStoreFile/915612316d6a09fe13e83a17e2d06051/cf2 2023-02-09 14:54:27,159 INFO [StoreOpener-915612316d6a09fe13e83a17e2d06051-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 915612316d6a09fe13e83a17e2d06051 columnFamilyName cf2 2023-02-09 14:54:27,159 INFO [StoreOpener-915612316d6a09fe13e83a17e2d06051-1] regionserver.HStore(310): Store=915612316d6a09fe13e83a17e2d06051/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:27,160 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionNoStoreFile/915612316d6a09fe13e83a17e2d06051 2023-02-09 14:54:27,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionNoStoreFile/915612316d6a09fe13e83a17e2d06051 2023-02-09 14:54:27,162 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:54:27,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 915612316d6a09fe13e83a17e2d06051 2023-02-09 14:54:27,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 915612316d6a09fe13e83a17e2d06051; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59649344, jitterRate=-0.11115550994873047}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:54:27,165 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 915612316d6a09fe13e83a17e2d06051: Running coprocessor pre-open hook at 1675954467154Writing region info on filesystem at 1675954467154Initializing all the Stores at 1675954467154Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954467155 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954467155Cleaning up temporary data from old regions at 1675954467161 (+6 ms)Cleaning up detritus from prior splits at 1675954467162 (+1 ms)Running coprocessor post-open hooks at 1675954467164 (+2 ms)Region opened successfully at 1675954467164 2023-02-09 14:54:27,165 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1675954466236.915612316d6a09fe13e83a17e2d06051., pid=62, masterSystemTime=1675954467150 2023-02-09 14:54:27,167 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1675954466236.915612316d6a09fe13e83a17e2d06051. 2023-02-09 14:54:27,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1675954466236.915612316d6a09fe13e83a17e2d06051. 2023-02-09 14:54:27,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,41,1675954466236.2c62e1c084902d2aa62c5102c5ff76cd. 2023-02-09 14:54:27,167 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2c62e1c084902d2aa62c5102c5ff76cd, NAME => 'testSplitTableRegionNoStoreFile,41,1675954466236.2c62e1c084902d2aa62c5102c5ff76cd.', STARTKEY => '41', ENDKEY => ''} 2023-02-09 14:54:27,167 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=915612316d6a09fe13e83a17e2d06051, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:27,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:54:27,167 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1675954466236.915612316d6a09fe13e83a17e2d06051.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675954467167"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954467167"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954467167"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954467167"}]},"ts":"1675954467167"} 2023-02-09 14:54:27,167 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:54:27,167 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,41,1675954466236.2c62e1c084902d2aa62c5102c5ff76cd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:27,168 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:54:27,168 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:54:27,169 INFO [StoreOpener-2c62e1c084902d2aa62c5102c5ff76cd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:54:27,171 DEBUG [StoreOpener-2c62e1c084902d2aa62c5102c5ff76cd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionNoStoreFile/2c62e1c084902d2aa62c5102c5ff76cd/cf1 2023-02-09 14:54:27,171 DEBUG [StoreOpener-2c62e1c084902d2aa62c5102c5ff76cd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionNoStoreFile/2c62e1c084902d2aa62c5102c5ff76cd/cf1 2023-02-09 14:54:27,171 INFO [StoreOpener-2c62e1c084902d2aa62c5102c5ff76cd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2c62e1c084902d2aa62c5102c5ff76cd columnFamilyName cf1 2023-02-09 14:54:27,171 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=59 2023-02-09 14:54:27,171 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=59, state=SUCCESS; OpenRegionProcedure 915612316d6a09fe13e83a17e2d06051, server=jenkins-hbase4.apache.org,33423,1675954443133 in 167 msec 2023-02-09 14:54:27,172 INFO [StoreOpener-2c62e1c084902d2aa62c5102c5ff76cd-1] regionserver.HStore(310): Store=2c62e1c084902d2aa62c5102c5ff76cd/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:27,172 INFO [StoreOpener-2c62e1c084902d2aa62c5102c5ff76cd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:54:27,173 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=915612316d6a09fe13e83a17e2d06051, ASSIGN in 328 msec 2023-02-09 14:54:27,174 DEBUG [StoreOpener-2c62e1c084902d2aa62c5102c5ff76cd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionNoStoreFile/2c62e1c084902d2aa62c5102c5ff76cd/cf2 2023-02-09 14:54:27,174 DEBUG [StoreOpener-2c62e1c084902d2aa62c5102c5ff76cd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionNoStoreFile/2c62e1c084902d2aa62c5102c5ff76cd/cf2 2023-02-09 14:54:27,175 INFO [StoreOpener-2c62e1c084902d2aa62c5102c5ff76cd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2c62e1c084902d2aa62c5102c5ff76cd columnFamilyName cf2 2023-02-09 14:54:27,175 INFO [StoreOpener-2c62e1c084902d2aa62c5102c5ff76cd-1] regionserver.HStore(310): Store=2c62e1c084902d2aa62c5102c5ff76cd/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:27,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionNoStoreFile/2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:54:27,177 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionNoStoreFile/2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:54:27,179 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:54:27,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:54:27,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 2c62e1c084902d2aa62c5102c5ff76cd; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59683053, jitterRate=-0.11065320670604706}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:54:27,182 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 2c62e1c084902d2aa62c5102c5ff76cd: Running coprocessor pre-open hook at 1675954467168Writing region info on filesystem at 1675954467168Initializing all the Stores at 1675954467168Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954467169 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954467169Cleaning up temporary data from old regions at 1675954467178 (+9 ms)Cleaning up detritus from prior splits at 1675954467178Running coprocessor post-open hooks at 1675954467182 (+4 ms)Region opened successfully at 1675954467182 2023-02-09 14:54:27,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1675954466236.2c62e1c084902d2aa62c5102c5ff76cd., pid=61, masterSystemTime=1675954467150 2023-02-09 14:54:27,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1675954466236.2c62e1c084902d2aa62c5102c5ff76cd. 2023-02-09 14:54:27,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,41,1675954466236.2c62e1c084902d2aa62c5102c5ff76cd. 2023-02-09 14:54:27,185 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=2c62e1c084902d2aa62c5102c5ff76cd, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:27,185 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1675954466236.2c62e1c084902d2aa62c5102c5ff76cd.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675954467184"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954467184"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954467184"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954467184"}]},"ts":"1675954467184"} 2023-02-09 14:54:27,189 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-02-09 14:54:27,189 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; OpenRegionProcedure 2c62e1c084902d2aa62c5102c5ff76cd, server=jenkins-hbase4.apache.org,33423,1675954443133 in 187 msec 2023-02-09 14:54:27,190 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=56 2023-02-09 14:54:27,191 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2c62e1c084902d2aa62c5102c5ff76cd, ASSIGN in 346 msec 2023-02-09 14:54:27,192 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7a3a2a2c1be8b68cf6b9c5576ca1d967, daughterA=915612316d6a09fe13e83a17e2d06051, daughterB=2c62e1c084902d2aa62c5102c5ff76cd in 955 msec 2023-02-09 14:54:27,241 DEBUG [Listener at localhost/40823] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 14:54:27,244 WARN [Listener at localhost/40823] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 14:54:27,244 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:27,244 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:27,245 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:27,245 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:27,245 INFO [Listener at localhost/40823] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-02-09 14:54:27,246 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionNoStoreFile 2023-02-09 14:54:27,246 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:54:27,248 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:27,248 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:27,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-09 14:54:27,249 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954467249"}]},"ts":"1675954467249"} 2023-02-09 14:54:27,250 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-02-09 14:54:27,253 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-02-09 14:54:27,254 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=915612316d6a09fe13e83a17e2d06051, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2c62e1c084902d2aa62c5102c5ff76cd, UNASSIGN}] 2023-02-09 14:54:27,255 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2c62e1c084902d2aa62c5102c5ff76cd, UNASSIGN 2023-02-09 14:54:27,255 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=2c62e1c084902d2aa62c5102c5ff76cd, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:27,256 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1675954466236.2c62e1c084902d2aa62c5102c5ff76cd.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675954467255"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954467255"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954467255"}]},"ts":"1675954467255"} 2023-02-09 14:54:27,257 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure 2c62e1c084902d2aa62c5102c5ff76cd, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:54:27,259 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=915612316d6a09fe13e83a17e2d06051, UNASSIGN 2023-02-09 14:54:27,260 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=915612316d6a09fe13e83a17e2d06051, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:27,260 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1675954466236.915612316d6a09fe13e83a17e2d06051.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675954467260"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954467260"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954467260"}]},"ts":"1675954467260"} 2023-02-09 14:54:27,261 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 915612316d6a09fe13e83a17e2d06051, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:54:27,349 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:27,349 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:27,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-09 14:54:27,409 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:54:27,410 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 2c62e1c084902d2aa62c5102c5ff76cd, disabling compactions & flushes 2023-02-09 14:54:27,410 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,41,1675954466236.2c62e1c084902d2aa62c5102c5ff76cd. 2023-02-09 14:54:27,410 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1675954466236.2c62e1c084902d2aa62c5102c5ff76cd. 2023-02-09 14:54:27,410 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,41,1675954466236.2c62e1c084902d2aa62c5102c5ff76cd. after waiting 0 ms 2023-02-09 14:54:27,410 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,41,1675954466236.2c62e1c084902d2aa62c5102c5ff76cd. 2023-02-09 14:54:27,414 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionNoStoreFile/2c62e1c084902d2aa62c5102c5ff76cd/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-09 14:54:27,414 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:54:27,414 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,41,1675954466236.2c62e1c084902d2aa62c5102c5ff76cd. 2023-02-09 14:54:27,414 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 2c62e1c084902d2aa62c5102c5ff76cd: Waiting for close lock at 1675954467409Running coprocessor pre-close hooks at 1675954467409Disabling compacts and flushes for region at 1675954467409Disabling writes for close at 1675954467410 (+1 ms)Writing region close event to WAL at 1675954467411 (+1 ms)Running coprocessor post-close hooks at 1675954467414 (+3 ms)Closed at 1675954467414 2023-02-09 14:54:27,416 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:54:27,416 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 915612316d6a09fe13e83a17e2d06051 2023-02-09 14:54:27,417 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 915612316d6a09fe13e83a17e2d06051, disabling compactions & flushes 2023-02-09 14:54:27,417 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1675954466236.915612316d6a09fe13e83a17e2d06051. 2023-02-09 14:54:27,417 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1675954466236.915612316d6a09fe13e83a17e2d06051. 2023-02-09 14:54:27,417 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1675954466236.915612316d6a09fe13e83a17e2d06051. after waiting 0 ms 2023-02-09 14:54:27,417 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1675954466236.915612316d6a09fe13e83a17e2d06051. 2023-02-09 14:54:27,417 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=2c62e1c084902d2aa62c5102c5ff76cd, regionState=CLOSED 2023-02-09 14:54:27,417 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1675954466236.2c62e1c084902d2aa62c5102c5ff76cd.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675954467417"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954467417"}]},"ts":"1675954467417"} 2023-02-09 14:54:27,421 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-09 14:54:27,421 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionNoStoreFile/915612316d6a09fe13e83a17e2d06051/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-09 14:54:27,421 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure 2c62e1c084902d2aa62c5102c5ff76cd, server=jenkins-hbase4.apache.org,33423,1675954443133 in 162 msec 2023-02-09 14:54:27,422 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:54:27,422 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1675954466236.915612316d6a09fe13e83a17e2d06051. 2023-02-09 14:54:27,422 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 915612316d6a09fe13e83a17e2d06051: Waiting for close lock at 1675954467416Running coprocessor pre-close hooks at 1675954467416Disabling compacts and flushes for region at 1675954467416Disabling writes for close at 1675954467417 (+1 ms)Writing region close event to WAL at 1675954467417Running coprocessor post-close hooks at 1675954467422 (+5 ms)Closed at 1675954467422 2023-02-09 14:54:27,422 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=2c62e1c084902d2aa62c5102c5ff76cd, UNASSIGN in 167 msec 2023-02-09 14:54:27,423 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 915612316d6a09fe13e83a17e2d06051 2023-02-09 14:54:27,423 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=915612316d6a09fe13e83a17e2d06051, regionState=CLOSED 2023-02-09 14:54:27,424 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1675954466236.915612316d6a09fe13e83a17e2d06051.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675954467423"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954467423"}]},"ts":"1675954467423"} 2023-02-09 14:54:27,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-02-09 14:54:27,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 915612316d6a09fe13e83a17e2d06051, server=jenkins-hbase4.apache.org,33423,1675954443133 in 164 msec 2023-02-09 14:54:27,427 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=64, resume processing ppid=63 2023-02-09 14:54:27,427 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=915612316d6a09fe13e83a17e2d06051, UNASSIGN in 172 msec 2023-02-09 14:54:27,428 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954467428"}]},"ts":"1675954467428"} 2023-02-09 14:54:27,430 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-02-09 14:54:27,431 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-02-09 14:54:27,433 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 186 msec 2023-02-09 14:54:27,551 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:27,551 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:27,552 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-09 14:54:27,552 INFO [Listener at localhost/40823] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-02-09 14:54:27,552 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:27,552 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:27,553 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionNoStoreFile 2023-02-09 14:54:27,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:54:27,555 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:54:27,556 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:27,556 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:27,556 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:54:27,556 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-09 14:54:27,561 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionNoStoreFile/7a3a2a2c1be8b68cf6b9c5576ca1d967 2023-02-09 14:54:27,561 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionNoStoreFile/2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:54:27,561 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionNoStoreFile/915612316d6a09fe13e83a17e2d06051 2023-02-09 14:54:27,563 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionNoStoreFile/915612316d6a09fe13e83a17e2d06051/cf1, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionNoStoreFile/915612316d6a09fe13e83a17e2d06051/cf2, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionNoStoreFile/915612316d6a09fe13e83a17e2d06051/recovered.edits] 2023-02-09 14:54:27,563 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionNoStoreFile/2c62e1c084902d2aa62c5102c5ff76cd/cf1, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionNoStoreFile/2c62e1c084902d2aa62c5102c5ff76cd/cf2, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionNoStoreFile/2c62e1c084902d2aa62c5102c5ff76cd/recovered.edits] 2023-02-09 14:54:27,563 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionNoStoreFile/7a3a2a2c1be8b68cf6b9c5576ca1d967/cf1, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionNoStoreFile/7a3a2a2c1be8b68cf6b9c5576ca1d967/cf2, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionNoStoreFile/7a3a2a2c1be8b68cf6b9c5576ca1d967/recovered.edits] 2023-02-09 14:54:27,568 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionNoStoreFile/2c62e1c084902d2aa62c5102c5ff76cd/recovered.edits/7.seqid to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionNoStoreFile/2c62e1c084902d2aa62c5102c5ff76cd/recovered.edits/7.seqid 2023-02-09 14:54:27,569 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionNoStoreFile/915612316d6a09fe13e83a17e2d06051/recovered.edits/7.seqid to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionNoStoreFile/915612316d6a09fe13e83a17e2d06051/recovered.edits/7.seqid 2023-02-09 14:54:27,569 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionNoStoreFile/2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:54:27,569 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionNoStoreFile/915612316d6a09fe13e83a17e2d06051 2023-02-09 14:54:27,569 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionNoStoreFile/7a3a2a2c1be8b68cf6b9c5576ca1d967/recovered.edits/4.seqid to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionNoStoreFile/7a3a2a2c1be8b68cf6b9c5576ca1d967/recovered.edits/4.seqid 2023-02-09 14:54:27,570 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionNoStoreFile/7a3a2a2c1be8b68cf6b9c5576ca1d967 2023-02-09 14:54:27,570 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-09 14:54:27,572 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:54:27,574 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-02-09 14:54:27,576 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-02-09 14:54:27,576 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:54:27,577 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-02-09 14:54:27,577 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1675954465733.7a3a2a2c1be8b68cf6b9c5576ca1d967.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954467577"}]},"ts":"9223372036854775807"} 2023-02-09 14:54:27,577 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1675954466236.915612316d6a09fe13e83a17e2d06051.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954467577"}]},"ts":"9223372036854775807"} 2023-02-09 14:54:27,577 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1675954466236.2c62e1c084902d2aa62c5102c5ff76cd.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954467577"}]},"ts":"9223372036854775807"} 2023-02-09 14:54:27,578 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 14:54:27,578 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 7a3a2a2c1be8b68cf6b9c5576ca1d967, NAME => 'testSplitTableRegionNoStoreFile,,1675954465733.7a3a2a2c1be8b68cf6b9c5576ca1d967.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 915612316d6a09fe13e83a17e2d06051, NAME => 'testSplitTableRegionNoStoreFile,,1675954466236.915612316d6a09fe13e83a17e2d06051.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 2c62e1c084902d2aa62c5102c5ff76cd, NAME => 'testSplitTableRegionNoStoreFile,41,1675954466236.2c62e1c084902d2aa62c5102c5ff76cd.', STARTKEY => '41', ENDKEY => ''}] 2023-02-09 14:54:27,578 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-02-09 14:54:27,579 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675954467578"}]},"ts":"9223372036854775807"} 2023-02-09 14:54:27,580 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-02-09 14:54:27,581 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:54:27,582 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 28 msec 2023-02-09 14:54:27,657 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:27,657 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:27,658 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-09 14:54:27,658 INFO [Listener at localhost/40823] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-02-09 14:54:27,672 INFO [Listener at localhost/40823] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=402 (was 397) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-296251100_15 at /127.0.0.1:54430 [Waiting for operation #27] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-296251100_15 at /127.0.0.1:42050 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=855 (was 853) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=237 (was 237), ProcessCount=168 (was 168), AvailableMemoryMB=3214 (was 3223) 2023-02-09 14:54:27,687 INFO [Listener at localhost/40823] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=402, OpenFileDescriptor=855, MaxFileDescriptor=60000, SystemLoadAverage=237, ProcessCount=168, AvailableMemoryMB=3213 2023-02-09 14:54:27,687 WARN [Listener at localhost/40823] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 14:54:27,688 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:27,688 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:27,690 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 14:54:27,690 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 14:54:27,692 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:27,693 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 14:54:27,694 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 14:54:27,695 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:27,696 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78 empty. 2023-02-09 14:54:27,696 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:27,696 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-09 14:54:27,710 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-02-09 14:54:27,711 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 94ffc82cbf109e7f71870e19837a0e78, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp 2023-02-09 14:54:27,725 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:27,726 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1603): Closing 94ffc82cbf109e7f71870e19837a0e78, disabling compactions & flushes 2023-02-09 14:54:27,726 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:27,726 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:27,726 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. after waiting 0 ms 2023-02-09 14:54:27,726 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:27,726 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:27,726 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 94ffc82cbf109e7f71870e19837a0e78: Waiting for close lock at 1675954467726Disabling compacts and flushes for region at 1675954467726Disabling writes for close at 1675954467726Writing region close event to WAL at 1675954467726Closed at 1675954467726 2023-02-09 14:54:27,729 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 14:54:27,730 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675954467730"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954467730"}]},"ts":"1675954467730"} 2023-02-09 14:54:27,731 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 14:54:27,732 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 14:54:27,733 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954467733"}]},"ts":"1675954467733"} 2023-02-09 14:54:27,734 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-02-09 14:54:27,738 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 14:54:27,739 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 14:54:27,739 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 14:54:27,739 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 14:54:27,739 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 14:54:27,739 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=94ffc82cbf109e7f71870e19837a0e78, ASSIGN}] 2023-02-09 14:54:27,741 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=94ffc82cbf109e7f71870e19837a0e78, ASSIGN 2023-02-09 14:54:27,742 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=94ffc82cbf109e7f71870e19837a0e78, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,41077,1675954443212; forceNewPlan=false, retain=false 2023-02-09 14:54:27,892 INFO [jenkins-hbase4:43829] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 14:54:27,893 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=94ffc82cbf109e7f71870e19837a0e78, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:27,893 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675954467893"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954467893"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954467893"}]},"ts":"1675954467893"} 2023-02-09 14:54:27,895 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure 94ffc82cbf109e7f71870e19837a0e78, server=jenkins-hbase4.apache.org,41077,1675954443212}] 2023-02-09 14:54:28,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:28,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 94ffc82cbf109e7f71870e19837a0e78, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:54:28,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:54:28,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:28,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:28,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:28,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:28,053 INFO [StoreOpener-94ffc82cbf109e7f71870e19837a0e78-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:28,054 DEBUG [StoreOpener-94ffc82cbf109e7f71870e19837a0e78-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1 2023-02-09 14:54:28,054 DEBUG [StoreOpener-94ffc82cbf109e7f71870e19837a0e78-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1 2023-02-09 14:54:28,054 INFO [StoreOpener-94ffc82cbf109e7f71870e19837a0e78-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 94ffc82cbf109e7f71870e19837a0e78 columnFamilyName cf1 2023-02-09 14:54:28,055 INFO [StoreOpener-94ffc82cbf109e7f71870e19837a0e78-1] regionserver.HStore(310): Store=94ffc82cbf109e7f71870e19837a0e78/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:28,055 INFO [StoreOpener-94ffc82cbf109e7f71870e19837a0e78-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:28,056 DEBUG [StoreOpener-94ffc82cbf109e7f71870e19837a0e78-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2 2023-02-09 14:54:28,057 DEBUG [StoreOpener-94ffc82cbf109e7f71870e19837a0e78-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2 2023-02-09 14:54:28,057 INFO [StoreOpener-94ffc82cbf109e7f71870e19837a0e78-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 94ffc82cbf109e7f71870e19837a0e78 columnFamilyName cf2 2023-02-09 14:54:28,057 INFO [StoreOpener-94ffc82cbf109e7f71870e19837a0e78-1] regionserver.HStore(310): Store=94ffc82cbf109e7f71870e19837a0e78/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:28,058 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:28,058 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:28,061 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:54:28,062 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:28,064 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:54:28,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 94ffc82cbf109e7f71870e19837a0e78; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71596243, jitterRate=0.0668671578168869}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:54:28,064 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 94ffc82cbf109e7f71870e19837a0e78: Running coprocessor pre-open hook at 1675954468051Writing region info on filesystem at 1675954468051Initializing all the Stores at 1675954468052 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954468052Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954468052Cleaning up temporary data from old regions at 1675954468059 (+7 ms)Cleaning up detritus from prior splits at 1675954468059Running coprocessor post-open hooks at 1675954468064 (+5 ms)Region opened successfully at 1675954468064 2023-02-09 14:54:28,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78., pid=71, masterSystemTime=1675954468047 2023-02-09 14:54:28,067 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:28,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:28,067 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=94ffc82cbf109e7f71870e19837a0e78, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,068 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675954468067"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954468067"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954468067"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954468067"}]},"ts":"1675954468067"} 2023-02-09 14:54:28,070 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-02-09 14:54:28,071 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure 94ffc82cbf109e7f71870e19837a0e78, server=jenkins-hbase4.apache.org,41077,1675954443212 in 174 msec 2023-02-09 14:54:28,072 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-02-09 14:54:28,072 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=94ffc82cbf109e7f71870e19837a0e78, ASSIGN in 332 msec 2023-02-09 14:54:28,073 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 14:54:28,073 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954468073"}]},"ts":"1675954468073"} 2023-02-09 14:54:28,074 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-02-09 14:54:28,077 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 14:54:28,078 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 386 msec 2023-02-09 14:54:28,201 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:28,201 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:28,202 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:28,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,204 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 14:54:28,204 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:28,204 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 14:54:28,204 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 14:54:28,204 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,204 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,206 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,206 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,206 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,206 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,206 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,206 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:28,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,208 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,208 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:28,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,208 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:28,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,208 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,209 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,209 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,209 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:28,209 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:28,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:28,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,209 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,209 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,209 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:28,209 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:28,209 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:28,209 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:28,210 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:28,209 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,210 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:28,210 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:28,210 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:28,210 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:28,210 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:28,210 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:28,212 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,212 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,212 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,212 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:28,212 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,212 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:28,215 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:28,215 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:28,215 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:28,215 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. started... 2023-02-09 14:54:28,216 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 94ffc82cbf109e7f71870e19837a0e78 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 14:54:28,228 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf1/6e24b414c2ba47f1bd35e83f1e758e6d 2023-02-09 14:54:28,252 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf2/6adf8d085dcc438aba69554d90273646 2023-02-09 14:54:28,259 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf1/6e24b414c2ba47f1bd35e83f1e758e6d as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/6e24b414c2ba47f1bd35e83f1e758e6d 2023-02-09 14:54:28,265 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/6e24b414c2ba47f1bd35e83f1e758e6d, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 14:54:28,266 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf2/6adf8d085dcc438aba69554d90273646 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/6adf8d085dcc438aba69554d90273646 2023-02-09 14:54:28,271 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/6adf8d085dcc438aba69554d90273646, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 14:54:28,272 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 94ffc82cbf109e7f71870e19837a0e78 in 56ms, sequenceid=6, compaction requested=false 2023-02-09 14:54:28,272 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 14:54:28,273 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 94ffc82cbf109e7f71870e19837a0e78: 2023-02-09 14:54:28,273 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:28,273 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:28,273 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:28,273 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,273 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:28,273 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:28,275 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,275 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:28,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:28,276 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:28,276 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:28,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:28,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:28,279 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 14:54:28,279 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@17e37fb0[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:28,279 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 14:54:28,279 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,280 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,280 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,280 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,281 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:28,281 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:28,281 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:28,281 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:28,281 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:28,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,281 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,281 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:28,281 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:28,281 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:28,281 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:28,281 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:28,282 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:28,282 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:28,283 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:28,284 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:28,284 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:28,284 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:28,285 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,285 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:28,285 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:28,285 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:28,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:28,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:28,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:28,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:28,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:28,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41077,1675954443212': 2023-02-09 14:54:28,289 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 14:54:28,289 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 14:54:28,289 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:28,289 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:28,289 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,289 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:28,291 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,291 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,291 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:28,291 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:28,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:28,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,291 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:28,291 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,291 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:28,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:28,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:28,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:28,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:28,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:28,292 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:28,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:28,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,293 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:28,293 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:28,296 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,296 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,296 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,303 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,303 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:28,303 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,303 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,303 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,304 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:28,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,303 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,304 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:28,304 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:28,304 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 14:54:28,304 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:28,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,304 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:28,305 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:28,303 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:28,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:28,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:28,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:28,305 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,306 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,306 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,306 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,306 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,306 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,306 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,306 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:28,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:28,405 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:28,405 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:28,405 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:28,406 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:28,419 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:28,419 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:28,420 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:28,422 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:28,422 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,422 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 14:54:28,422 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:28,422 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 14:54:28,422 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 14:54:28,422 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,422 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,423 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,424 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,424 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,424 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,424 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,424 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:28,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,424 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,424 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,425 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:28,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:28,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:28,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,425 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,425 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,425 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,425 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 14:54:28,425 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,425 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 14:54:28,425 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,425 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:28,425 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 14:54:28,425 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:28,425 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,425 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:28,426 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:28,426 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:28,426 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:28,426 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:28,426 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:28,426 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:28,426 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:28,426 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:28,426 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:28,426 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:28,426 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:28,426 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:28,426 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:28,426 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. started... 2023-02-09 14:54:28,426 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 94ffc82cbf109e7f71870e19837a0e78 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:54:28,427 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,427 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,428 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,428 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,428 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:28,428 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:28,439 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf1/60125f034d6a4b318f6a5e94a304c87c 2023-02-09 14:54:28,454 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf2/9974ec9e406648e58660186bb1d7f420 2023-02-09 14:54:28,458 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf1/60125f034d6a4b318f6a5e94a304c87c as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/60125f034d6a4b318f6a5e94a304c87c 2023-02-09 14:54:28,462 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/60125f034d6a4b318f6a5e94a304c87c, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 14:54:28,462 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf2/9974ec9e406648e58660186bb1d7f420 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/9974ec9e406648e58660186bb1d7f420 2023-02-09 14:54:28,466 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/9974ec9e406648e58660186bb1d7f420, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 14:54:28,467 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 94ffc82cbf109e7f71870e19837a0e78 in 41ms, sequenceid=19, compaction requested=false 2023-02-09 14:54:28,467 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 94ffc82cbf109e7f71870e19837a0e78: 2023-02-09 14:54:28,467 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:28,467 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:28,467 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:28,467 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,467 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:28,467 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:28,469 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,469 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:28,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:28,469 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,469 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:28,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:28,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:28,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:28,471 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 14:54:28,471 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2e992627[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:28,471 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 14:54:28,471 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,473 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,473 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,473 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,473 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,474 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:28,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,473 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:28,474 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:28,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,474 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:28,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,474 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:28,474 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:28,474 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:28,474 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:28,474 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:28,474 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:28,475 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:28,475 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:28,475 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:28,476 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:28,476 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:28,476 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,476 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:28,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,476 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:28,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:28,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:28,476 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:28,476 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:28,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:28,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:28,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:28,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41077,1675954443212': 2023-02-09 14:54:28,479 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 14:54:28,479 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 14:54:28,479 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:28,479 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:28,479 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,479 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:28,481 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,481 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:28,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:28,481 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:28,481 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,481 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,481 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:28,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,481 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:28,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:28,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,482 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:28,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:28,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:28,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:28,483 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:28,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:28,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:28,484 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:28,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,485 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,486 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,486 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,490 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,490 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,490 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,490 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,490 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:28,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,490 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:28,490 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,490 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:28,490 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,491 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 14:54:28,491 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:28,491 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:28,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:28,490 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:28,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:28,491 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:28,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,491 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:28,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:28,491 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,491 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,491 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,491 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:28,492 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:28,492 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,492 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,591 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:28,591 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:28,591 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:28,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:28,604 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:28,604 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:28,604 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:28,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:28,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,606 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 14:54:28,606 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:28,606 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 14:54:28,606 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 14:54:28,607 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,607 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,608 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,608 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,608 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,608 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,608 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,608 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:28,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,609 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,609 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,609 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:28,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,609 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:28,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,609 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:28,610 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 14:54:28,610 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 14:54:28,610 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,610 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:28,610 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:28,610 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,610 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:28,610 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:28,610 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:28,610 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:28,610 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:28,610 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,610 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:28,610 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:28,610 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:28,610 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:28,610 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. started... 2023-02-09 14:54:28,610 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 94ffc82cbf109e7f71870e19837a0e78 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:54:28,611 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,612 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,612 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:28,616 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 14:54:28,616 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,616 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:28,616 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:28,616 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:28,616 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:28,616 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:28,617 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,618 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,618 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:28,627 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf1/b24f5d9989ca4bb7945b363f3bc2a65a 2023-02-09 14:54:28,640 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf2/39a737174efa406f8685bd11869876a4 2023-02-09 14:54:28,644 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf1/b24f5d9989ca4bb7945b363f3bc2a65a as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/b24f5d9989ca4bb7945b363f3bc2a65a 2023-02-09 14:54:28,648 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/b24f5d9989ca4bb7945b363f3bc2a65a, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 14:54:28,649 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf2/39a737174efa406f8685bd11869876a4 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/39a737174efa406f8685bd11869876a4 2023-02-09 14:54:28,653 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/39a737174efa406f8685bd11869876a4, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 14:54:28,654 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 94ffc82cbf109e7f71870e19837a0e78 in 44ms, sequenceid=32, compaction requested=true 2023-02-09 14:54:28,654 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 94ffc82cbf109e7f71870e19837a0e78: 2023-02-09 14:54:28,654 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:28,654 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:28,654 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:28,654 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,654 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:28,654 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:28,659 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,659 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:28,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:28,659 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,659 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:28,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:28,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:28,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:28,661 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 14:54:28,661 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1ee9773b[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:28,661 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 14:54:28,661 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,662 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,662 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,662 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,663 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:28,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,663 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,663 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:28,663 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:28,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,663 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:28,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,663 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:28,663 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:28,663 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:28,663 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:28,663 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:28,663 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:28,664 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:28,664 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:28,664 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,664 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:28,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:28,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:28,666 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:28,666 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:28,666 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:28,666 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:28,666 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:28,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:28,666 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:28,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:28,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:28,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41077,1675954443212': 2023-02-09 14:54:28,669 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 14:54:28,669 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 14:54:28,669 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:28,669 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:28,669 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,669 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:28,671 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,671 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,671 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,671 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:28,671 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:28,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:28,671 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:28,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:28,671 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:28,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:28,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:28,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:28,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,671 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:28,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:28,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:28,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,672 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,672 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:28,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:28,674 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,674 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,674 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,676 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,676 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,676 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,677 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,677 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,677 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:28,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,677 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:28,677 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:28,677 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,676 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,677 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,677 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 14:54:28,677 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:28,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,677 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,677 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:28,677 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:28,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:28,677 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:28,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:28,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:28,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:28,677 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,677 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:28,678 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:28,678 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,678 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,778 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:28,778 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:28,778 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:28,779 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:28,792 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:28,792 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:28,793 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:28,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:28,795 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,795 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 14:54:28,795 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:28,795 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 14:54:28,795 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 14:54:28,795 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,795 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,798 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,798 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,798 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,798 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,798 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,798 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:28,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,799 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,799 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,799 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:28,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,799 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:28,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,799 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,799 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 14:54:28,799 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:28,800 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:28,800 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:28,799 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 14:54:28,800 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:28,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,800 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:28,800 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,800 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:28,800 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,800 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:28,800 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:28,800 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:28,800 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 14:54:28,800 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,800 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:28,800 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:28,800 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:28,800 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:28,801 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:28,801 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. started... 2023-02-09 14:54:28,801 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:28,801 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:28,801 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:28,801 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 94ffc82cbf109e7f71870e19837a0e78 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:54:28,801 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,802 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,802 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:28,802 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,803 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,803 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:28,811 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf1/bec1a1460d3648ad83b9b6db4ce4809e 2023-02-09 14:54:28,823 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf2/aaf025f0f70e4a3e839770e623098e97 2023-02-09 14:54:28,827 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf1/bec1a1460d3648ad83b9b6db4ce4809e as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/bec1a1460d3648ad83b9b6db4ce4809e 2023-02-09 14:54:28,831 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/bec1a1460d3648ad83b9b6db4ce4809e, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 14:54:28,832 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf2/aaf025f0f70e4a3e839770e623098e97 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/aaf025f0f70e4a3e839770e623098e97 2023-02-09 14:54:28,835 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/aaf025f0f70e4a3e839770e623098e97, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 14:54:28,836 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 94ffc82cbf109e7f71870e19837a0e78 in 35ms, sequenceid=45, compaction requested=true 2023-02-09 14:54:28,836 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 94ffc82cbf109e7f71870e19837a0e78: 2023-02-09 14:54:28,836 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:28,837 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:28,837 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:28,837 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,837 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:28,837 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:28,838 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,838 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:28,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:28,838 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,838 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:28,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:28,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:28,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:28,840 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 14:54:28,841 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@375d4751[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:28,841 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 14:54:28,841 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,843 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,843 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,843 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,843 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:28,843 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:28,843 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,843 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:28,843 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:28,843 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:28,843 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:28,844 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:28,844 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:28,844 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:28,844 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:28,845 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:28,845 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,845 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:28,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:28,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:28,845 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:28,845 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:28,845 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:28,845 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:28,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:28,846 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:28,846 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:28,846 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:28,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:28,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:28,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41077,1675954443212': 2023-02-09 14:54:28,848 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 14:54:28,848 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 14:54:28,848 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:28,848 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:28,849 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,849 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:28,853 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,853 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,853 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,853 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:28,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:28,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:28,853 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,854 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:28,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:28,854 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,854 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:28,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:28,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:28,854 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:28,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:28,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:28,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:28,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,855 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:28,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,856 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:28,857 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,857 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,859 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,859 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,859 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,859 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,859 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,859 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,859 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:28,859 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,859 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 14:54:28,859 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:28,859 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:28,859 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:28,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,859 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,860 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,859 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:28,860 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:28,860 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:28,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:28,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:28,860 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:28,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,860 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:28,860 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,860 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,860 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:28,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:28,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:28,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:28,960 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:28,960 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:28,960 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:28,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:28,973 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:28,974 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:28,974 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:28,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:28,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,976 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 14:54:28,976 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:28,976 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 14:54:28,976 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 14:54:28,976 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,976 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,978 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,978 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,978 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,978 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:28,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:28,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:28,978 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:28,978 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:28,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,979 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,979 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,979 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:28,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:28,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,979 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:28,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,979 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,979 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,979 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 14:54:28,979 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,979 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 14:54:28,979 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:28,980 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:28,979 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 14:54:28,980 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:28,979 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,980 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,980 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:28,980 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:28,980 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:28,980 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:28,980 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:28,980 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:28,980 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:28,980 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:28,980 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:28,980 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. started... 2023-02-09 14:54:28,980 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:28,980 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:28,980 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:28,980 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:28,980 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 94ffc82cbf109e7f71870e19837a0e78 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:54:28,985 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,985 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,985 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,985 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:28,985 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:28,985 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:28,989 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf1/b2da05fec39f4cfd98dc4b8debb7657e 2023-02-09 14:54:29,000 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf2/803c92c516a94497bd8e320dd6fd8469 2023-02-09 14:54:29,004 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf1/b2da05fec39f4cfd98dc4b8debb7657e as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/b2da05fec39f4cfd98dc4b8debb7657e 2023-02-09 14:54:29,007 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/b2da05fec39f4cfd98dc4b8debb7657e, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 14:54:29,008 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf2/803c92c516a94497bd8e320dd6fd8469 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/803c92c516a94497bd8e320dd6fd8469 2023-02-09 14:54:29,011 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/803c92c516a94497bd8e320dd6fd8469, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 14:54:29,012 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 94ffc82cbf109e7f71870e19837a0e78 in 32ms, sequenceid=58, compaction requested=true 2023-02-09 14:54:29,012 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 94ffc82cbf109e7f71870e19837a0e78: 2023-02-09 14:54:29,012 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:29,013 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:29,013 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:29,013 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,013 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:29,013 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:29,014 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,014 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:29,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:29,015 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,015 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:29,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:29,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:29,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:29,016 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 14:54:29,016 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b4fb0f5[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:29,016 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 14:54:29,016 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,017 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,017 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,017 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,018 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:29,018 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:29,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,018 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:29,018 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:29,018 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:29,018 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,018 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:29,018 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:29,018 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:29,018 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:29,018 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:29,019 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:29,019 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:29,019 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:29,021 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:29,021 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:29,021 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,021 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:29,021 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:29,021 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:29,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:29,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:29,021 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:29,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:29,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:29,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:29,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41077,1675954443212': 2023-02-09 14:54:29,024 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 14:54:29,024 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 14:54:29,025 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:29,025 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:29,025 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,025 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:29,026 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,026 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:29,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:29,026 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,026 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,026 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,027 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:29,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,026 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,027 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,028 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,028 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:29,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,030 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:29,030 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,030 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,033 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,033 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,033 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,033 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,033 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,033 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,033 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:29,033 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,033 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,033 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 14:54:29,033 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:29,033 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,033 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,033 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,033 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,034 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:29,034 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:29,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,033 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,034 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,034 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,034 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,034 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,134 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:29,134 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:29,134 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:29,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:29,147 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:29,148 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:29,148 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:29,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:29,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,150 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 14:54:29,150 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:29,150 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 14:54:29,150 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 14:54:29,151 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,151 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,155 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,155 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,155 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,155 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,155 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,155 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:29,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,156 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,156 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,156 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:29,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,156 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:29,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:29,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,156 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,156 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 14:54:29,156 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,156 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,156 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:29,157 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 14:54:29,157 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:29,157 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,157 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:29,157 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 14:54:29,157 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:29,157 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,157 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:29,157 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:29,157 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:29,157 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:29,157 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:29,157 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:29,157 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:29,157 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:29,157 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:29,157 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:29,157 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:29,157 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:29,157 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. started... 2023-02-09 14:54:29,157 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 94ffc82cbf109e7f71870e19837a0e78 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:54:29,158 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,159 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,159 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,159 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:29,159 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,159 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:29,169 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf1/768ce0ae783f4044a3d4d153d4cfc4b9 2023-02-09 14:54:29,181 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf2/6c84bb9704ec4bfcbadfb6d74ba97d77 2023-02-09 14:54:29,185 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf1/768ce0ae783f4044a3d4d153d4cfc4b9 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/768ce0ae783f4044a3d4d153d4cfc4b9 2023-02-09 14:54:29,188 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/768ce0ae783f4044a3d4d153d4cfc4b9, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 14:54:29,189 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf2/6c84bb9704ec4bfcbadfb6d74ba97d77 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/6c84bb9704ec4bfcbadfb6d74ba97d77 2023-02-09 14:54:29,192 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/6c84bb9704ec4bfcbadfb6d74ba97d77, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 14:54:29,193 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 94ffc82cbf109e7f71870e19837a0e78 in 36ms, sequenceid=71, compaction requested=true 2023-02-09 14:54:29,193 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 94ffc82cbf109e7f71870e19837a0e78: 2023-02-09 14:54:29,193 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:29,193 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:29,193 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:29,193 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,193 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:29,193 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:29,196 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,196 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:29,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:29,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:29,197 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,197 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:29,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:29,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:29,198 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 14:54:29,198 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5c8011a5[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:29,199 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 14:54:29,199 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,200 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,200 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,200 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,200 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,200 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:29,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,200 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:29,200 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:29,200 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:29,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,200 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:29,200 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:29,200 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:29,200 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:29,200 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:29,200 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:29,202 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:29,202 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,202 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:29,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:29,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:29,202 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:29,202 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:29,202 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:29,202 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:29,202 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:29,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:29,202 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:29,202 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:29,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:29,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:29,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41077,1675954443212': 2023-02-09 14:54:29,205 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 14:54:29,205 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 14:54:29,205 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:29,205 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:29,205 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,205 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:29,207 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,207 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,207 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,207 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,207 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,207 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:29,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:29,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,207 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,208 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:29,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,208 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,209 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:29,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:29,210 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,210 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,211 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,213 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,213 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,213 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,213 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,213 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,213 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,213 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,213 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,213 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,213 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,213 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,214 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 14:54:29,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,213 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,214 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:29,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,214 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:29,214 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,214 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,214 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,214 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,214 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,214 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:29,215 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:29,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,315 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:29,315 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:29,315 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:29,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:29,327 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:29,327 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:29,328 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:29,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:29,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,329 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 14:54:29,330 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:29,330 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 14:54:29,330 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 14:54:29,330 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,330 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,331 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,331 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,331 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,331 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,332 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,332 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:29,332 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,332 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,332 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:29,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,332 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:29,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:29,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,333 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,333 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,333 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 14:54:29,333 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,333 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:29,333 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:29,333 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 14:54:29,333 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 14:54:29,333 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,333 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:29,333 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:29,333 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,333 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:29,333 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:29,333 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:29,333 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:29,333 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:29,333 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. started... 2023-02-09 14:54:29,333 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:29,333 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:29,333 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:29,333 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 94ffc82cbf109e7f71870e19837a0e78 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-02-09 14:54:29,333 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:29,333 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:29,334 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:29,333 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:29,335 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,335 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,335 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,335 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,335 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:29,335 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:29,343 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf1/1a39c4a25ff8458590b3f608aa05825a 2023-02-09 14:54:29,347 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1a39c4a25ff8458590b3f608aa05825a 2023-02-09 14:54:29,355 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf2/ca9fd58be3704739a1150ab8f4f79ea4 2023-02-09 14:54:29,358 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ca9fd58be3704739a1150ab8f4f79ea4 2023-02-09 14:54:29,359 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf1/1a39c4a25ff8458590b3f608aa05825a as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/1a39c4a25ff8458590b3f608aa05825a 2023-02-09 14:54:29,362 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1a39c4a25ff8458590b3f608aa05825a 2023-02-09 14:54:29,362 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/1a39c4a25ff8458590b3f608aa05825a, entries=9, sequenceid=83, filesize=5.1 K 2023-02-09 14:54:29,362 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf2/ca9fd58be3704739a1150ab8f4f79ea4 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/ca9fd58be3704739a1150ab8f4f79ea4 2023-02-09 14:54:29,366 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ca9fd58be3704739a1150ab8f4f79ea4 2023-02-09 14:54:29,366 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/ca9fd58be3704739a1150ab8f4f79ea4, entries=9, sequenceid=83, filesize=5.1 K 2023-02-09 14:54:29,367 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for 94ffc82cbf109e7f71870e19837a0e78 in 33ms, sequenceid=83, compaction requested=true 2023-02-09 14:54:29,367 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 94ffc82cbf109e7f71870e19837a0e78: 2023-02-09 14:54:29,367 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:29,367 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:29,367 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:29,367 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,367 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:29,367 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:29,370 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,370 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:29,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:29,370 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,370 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:29,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:29,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:29,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:29,373 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 14:54:29,373 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@905f5a1[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:29,373 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 14:54:29,373 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,374 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,374 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,374 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,375 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:29,375 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:29,375 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:29,375 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:29,375 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,375 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:29,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,375 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:29,375 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:29,375 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:29,375 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:29,375 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:29,377 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:29,377 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:29,377 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:29,382 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,382 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:29,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,382 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:29,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:29,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:29,382 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:29,382 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:29,382 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:29,382 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:29,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:29,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:29,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:29,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41077,1675954443212': 2023-02-09 14:54:29,386 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 14:54:29,387 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 14:54:29,387 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:29,387 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:29,387 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,387 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:29,388 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,388 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,388 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,388 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:29,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:29,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,388 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,388 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,389 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:29,389 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,389 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:29,390 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:29,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,392 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,392 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,392 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,395 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,395 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,395 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,395 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,395 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,395 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,395 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,396 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:29,396 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 14:54:29,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,395 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,396 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:29,396 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,396 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,396 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:29,396 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:29,396 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,396 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,396 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,396 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,396 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,397 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,397 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,496 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:29,496 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:29,497 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:29,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:29,505 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:29,505 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:29,506 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:29,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:29,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,507 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 14:54:29,507 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:29,507 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 14:54:29,507 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 14:54:29,508 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,508 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,509 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,509 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,509 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,509 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,509 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,510 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:29,510 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,510 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,510 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:29,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:29,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:29,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,510 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,510 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,510 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,510 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 14:54:29,511 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,511 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 14:54:29,511 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:29,511 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,511 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 14:54:29,511 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:29,511 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:29,511 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,511 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:29,511 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:29,511 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:29,511 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:29,511 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:29,511 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:29,511 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:29,511 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:29,511 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:29,511 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:29,511 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:29,511 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:29,511 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:29,511 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. started... 2023-02-09 14:54:29,511 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 94ffc82cbf109e7f71870e19837a0e78 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-09 14:54:29,513 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,513 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,513 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,514 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:29,514 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,514 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:29,520 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf1/d1e0c6c3f72940668e505a5227f8b1bb 2023-02-09 14:54:29,550 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d1e0c6c3f72940668e505a5227f8b1bb 2023-02-09 14:54:29,562 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf2/92a9fcb53ed146f58cd20ee6828e019a 2023-02-09 14:54:29,566 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 92a9fcb53ed146f58cd20ee6828e019a 2023-02-09 14:54:29,567 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf1/d1e0c6c3f72940668e505a5227f8b1bb as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/d1e0c6c3f72940668e505a5227f8b1bb 2023-02-09 14:54:29,571 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d1e0c6c3f72940668e505a5227f8b1bb 2023-02-09 14:54:29,571 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/d1e0c6c3f72940668e505a5227f8b1bb, entries=5, sequenceid=91, filesize=5.0 K 2023-02-09 14:54:29,571 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf2/92a9fcb53ed146f58cd20ee6828e019a as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/92a9fcb53ed146f58cd20ee6828e019a 2023-02-09 14:54:29,574 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 92a9fcb53ed146f58cd20ee6828e019a 2023-02-09 14:54:29,574 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/92a9fcb53ed146f58cd20ee6828e019a, entries=5, sequenceid=91, filesize=5.0 K 2023-02-09 14:54:29,575 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 94ffc82cbf109e7f71870e19837a0e78 in 64ms, sequenceid=91, compaction requested=true 2023-02-09 14:54:29,575 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 94ffc82cbf109e7f71870e19837a0e78: 2023-02-09 14:54:29,575 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:29,575 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:29,575 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:29,575 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,576 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:29,576 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:29,577 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,577 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:29,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:29,577 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,577 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:29,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:29,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:29,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:29,579 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 14:54:29,579 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@543f5c1f[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:29,579 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 14:54:29,579 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,580 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,580 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,580 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,580 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:29,580 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:29,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,580 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:29,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,580 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,580 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:29,581 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:29,581 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:29,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,581 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:29,581 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:29,581 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:29,581 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:29,582 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:29,582 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:29,582 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:29,584 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,584 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:29,584 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:29,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:29,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:29,584 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:29,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:29,585 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:29,585 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:29,585 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:29,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:29,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:29,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41077,1675954443212': 2023-02-09 14:54:29,587 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 14:54:29,587 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 14:54:29,587 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:29,587 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:29,587 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,587 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:29,589 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,589 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,589 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,589 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,589 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,589 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:29,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:29,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,589 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,589 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:29,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,590 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,590 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:29,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,592 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:29,592 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,592 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,597 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,597 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,597 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,597 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,598 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:29,597 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,598 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 14:54:29,598 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:29,598 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,598 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,598 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:29,598 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,598 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:29,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,598 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,598 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,598 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,598 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,599 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,599 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,599 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,599 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,599 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,698 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:29,699 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:29,699 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:29,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:29,706 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:29,706 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:29,707 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:54:29,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 14:54:29,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,708 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 14:54:29,708 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:29,708 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-09 14:54:29,709 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-09 14:54:29,709 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,709 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,710 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,710 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,710 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,710 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,710 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:54:29,710 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,711 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,711 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:29,711 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:29,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,711 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,711 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:54:29,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,712 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,712 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 14:54:29,712 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,712 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 14:54:29,712 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-09 14:54:29,712 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:29,712 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,712 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,712 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:29,712 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-09 14:54:29,712 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:29,712 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:29,712 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:29,712 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:29,712 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:29,712 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:29,712 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:54:29,712 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:54:29,712 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:54:29,712 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:29,712 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-09 14:54:29,712 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:29,712 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. started... 2023-02-09 14:54:29,712 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:29,713 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 94ffc82cbf109e7f71870e19837a0e78 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-09 14:54:29,713 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,714 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,714 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,714 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:29,714 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,714 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:29,721 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf1/f267f4025fb34ba3ab5ddd61ba69810b 2023-02-09 14:54:29,725 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f267f4025fb34ba3ab5ddd61ba69810b 2023-02-09 14:54:29,732 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf2/7058f4e7029c4e35912b425b094d4fb4 2023-02-09 14:54:29,735 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7058f4e7029c4e35912b425b094d4fb4 2023-02-09 14:54:29,736 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf1/f267f4025fb34ba3ab5ddd61ba69810b as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/f267f4025fb34ba3ab5ddd61ba69810b 2023-02-09 14:54:29,740 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f267f4025fb34ba3ab5ddd61ba69810b 2023-02-09 14:54:29,740 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/f267f4025fb34ba3ab5ddd61ba69810b, entries=5, sequenceid=99, filesize=5.0 K 2023-02-09 14:54:29,741 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf2/7058f4e7029c4e35912b425b094d4fb4 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/7058f4e7029c4e35912b425b094d4fb4 2023-02-09 14:54:29,744 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7058f4e7029c4e35912b425b094d4fb4 2023-02-09 14:54:29,744 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/7058f4e7029c4e35912b425b094d4fb4, entries=5, sequenceid=99, filesize=5.0 K 2023-02-09 14:54:29,745 INFO [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 94ffc82cbf109e7f71870e19837a0e78 in 33ms, sequenceid=99, compaction requested=true 2023-02-09 14:54:29,745 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 94ffc82cbf109e7f71870e19837a0e78: 2023-02-09 14:54:29,745 DEBUG [rs(jenkins-hbase4.apache.org,41077,1675954443212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:29,745 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:54:29,745 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:54:29,745 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,745 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-09 14:54:29,746 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-09 14:54:29,752 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,752 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:29,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:29,752 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,753 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:54:29,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:29,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:29,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:29,754 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-09 14:54:29,754 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@12318675[Count = 0] remaining members to acquire global barrier 2023-02-09 14:54:29,754 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-09 14:54:29,754 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,755 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,755 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,755 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,756 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:29,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,756 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:29,756 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,756 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:54:29,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,756 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:29,756 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-09 14:54:29,756 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:29,756 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:54:29,756 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:54:29,756 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-09 14:54:29,756 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:54:29,757 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,757 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:29,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:29,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:29,757 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:29,757 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:29,757 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:29,757 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:29,757 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:29,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:29,758 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-09 14:54:29,758 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:29,758 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-09 14:54:29,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:29,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:29,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,41077,1675954443212': 2023-02-09 14:54:29,760 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-09 14:54:29,760 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-09 14:54:29,760 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:54:29,760 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:54:29,760 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,760 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:54:29,763 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,763 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:54:29,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:54:29,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,763 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,763 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,763 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:54:29,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,763 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,763 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,763 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,764 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,764 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:54:29,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,765 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:54:29,766 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,766 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,768 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,768 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,768 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,768 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,768 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,768 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,768 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:54:29,768 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,768 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-09 14:54:29,768 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:54:29,768 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,768 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:54:29,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:54:29,768 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:54:29,768 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:54:29,768 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:54:29,769 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 14:54:29,769 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:54:29,769 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,769 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:54:29,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:54:29,769 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:29,769 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,769 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,769 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:29,869 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:54:29,869 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:29,869 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:29,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:54:29,872 DEBUG [Listener at localhost/40823] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,873 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=94ffc82cbf109e7f71870e19837a0e78, daughterA=c720e43e6c74641ba50283474b45a523, daughterB=8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:54:29,874 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=94ffc82cbf109e7f71870e19837a0e78, daughterA=c720e43e6c74641ba50283474b45a523, daughterB=8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:54:29,874 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=94ffc82cbf109e7f71870e19837a0e78, daughterA=c720e43e6c74641ba50283474b45a523, daughterB=8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:54:29,875 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=94ffc82cbf109e7f71870e19837a0e78, daughterA=c720e43e6c74641ba50283474b45a523, daughterB=8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:54:29,876 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=94ffc82cbf109e7f71870e19837a0e78, UNASSIGN}] 2023-02-09 14:54:29,877 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=94ffc82cbf109e7f71870e19837a0e78, UNASSIGN 2023-02-09 14:54:29,877 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=94ffc82cbf109e7f71870e19837a0e78, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:29,878 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675954469877"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954469877"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954469877"}]},"ts":"1675954469877"} 2023-02-09 14:54:29,879 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure 94ffc82cbf109e7f71870e19837a0e78, server=jenkins-hbase4.apache.org,41077,1675954443212}] 2023-02-09 14:54:30,031 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,031 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 94ffc82cbf109e7f71870e19837a0e78, disabling compactions & flushes 2023-02-09 14:54:30,031 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:30,031 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:30,031 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. after waiting 0 ms 2023-02-09 14:54:30,031 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:30,031 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 94ffc82cbf109e7f71870e19837a0e78 2/2 column families, dataSize=50 B heapSize=720 B 2023-02-09 14:54:30,041 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf1/6bc92d7db7c04f2092d0d357544803c0 2023-02-09 14:54:30,045 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6bc92d7db7c04f2092d0d357544803c0 2023-02-09 14:54:30,053 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf2/3259f9dfc059461f86903f033c01ca31 2023-02-09 14:54:30,056 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3259f9dfc059461f86903f033c01ca31 2023-02-09 14:54:30,057 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf1/6bc92d7db7c04f2092d0d357544803c0 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/6bc92d7db7c04f2092d0d357544803c0 2023-02-09 14:54:30,061 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6bc92d7db7c04f2092d0d357544803c0 2023-02-09 14:54:30,061 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/6bc92d7db7c04f2092d0d357544803c0, entries=1, sequenceid=103, filesize=4.9 K 2023-02-09 14:54:30,062 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.tmp/cf2/3259f9dfc059461f86903f033c01ca31 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/3259f9dfc059461f86903f033c01ca31 2023-02-09 14:54:30,065 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3259f9dfc059461f86903f033c01ca31 2023-02-09 14:54:30,065 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/3259f9dfc059461f86903f033c01ca31, entries=1, sequenceid=103, filesize=4.9 K 2023-02-09 14:54:30,066 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 94ffc82cbf109e7f71870e19837a0e78 in 35ms, sequenceid=103, compaction requested=true 2023-02-09 14:54:30,070 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-02-09 14:54:30,071 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:54:30,071 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. 2023-02-09 14:54:30,071 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 94ffc82cbf109e7f71870e19837a0e78: Waiting for close lock at 1675954470031Running coprocessor pre-close hooks at 1675954470031Disabling compacts and flushes for region at 1675954470031Disabling writes for close at 1675954470031Obtaining lock to block concurrent updates at 1675954470031Preparing flush snapshotting stores in 94ffc82cbf109e7f71870e19837a0e78 at 1675954470031Finished memstore snapshotting testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78., syncing WAL and waiting on mvcc, flushsize=dataSize=50, getHeapSize=688, getOffHeapSize=0, getCellsCount=2 at 1675954470032 (+1 ms)Flushing stores of testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78. at 1675954470032Flushing 94ffc82cbf109e7f71870e19837a0e78/cf1: creating writer at 1675954470033 (+1 ms)Flushing 94ffc82cbf109e7f71870e19837a0e78/cf1: appending metadata at 1675954470035 (+2 ms)Flushing 94ffc82cbf109e7f71870e19837a0e78/cf1: closing flushed file at 1675954470035Flushing 94ffc82cbf109e7f71870e19837a0e78/cf2: creating writer at 1675954470045 (+10 ms)Flushing 94ffc82cbf109e7f71870e19837a0e78/cf2: appending metadata at 1675954470047 (+2 ms)Flushing 94ffc82cbf109e7f71870e19837a0e78/cf2: closing flushed file at 1675954470047Flushing 94ffc82cbf109e7f71870e19837a0e78/cf1: reopening flushed file at 1675954470058 (+11 ms)Flushing 94ffc82cbf109e7f71870e19837a0e78/cf2: reopening flushed file at 1675954470062 (+4 ms)Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 94ffc82cbf109e7f71870e19837a0e78 in 35ms, sequenceid=103, compaction requested=true at 1675954470066 (+4 ms)Writing region close event to WAL at 1675954470068 (+2 ms)Running coprocessor post-close hooks at 1675954470071 (+3 ms)Closed at 1675954470071 2023-02-09 14:54:30,072 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,073 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=94ffc82cbf109e7f71870e19837a0e78, regionState=CLOSED 2023-02-09 14:54:30,073 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675954470073"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954470073"}]},"ts":"1675954470073"} 2023-02-09 14:54:30,075 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-02-09 14:54:30,075 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure 94ffc82cbf109e7f71870e19837a0e78, server=jenkins-hbase4.apache.org,41077,1675954443212 in 195 msec 2023-02-09 14:54:30,077 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-09 14:54:30,077 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=94ffc82cbf109e7f71870e19837a0e78, UNASSIGN in 200 msec 2023-02-09 14:54:30,092 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=94ffc82cbf109e7f71870e19837a0e78, threads=16 2023-02-09 14:54:30,092 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/3259f9dfc059461f86903f033c01ca31 for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,092 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/39a737174efa406f8685bd11869876a4 for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,093 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/6adf8d085dcc438aba69554d90273646 for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,093 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/6c84bb9704ec4bfcbadfb6d74ba97d77 for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,093 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/7058f4e7029c4e35912b425b094d4fb4 for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,094 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/803c92c516a94497bd8e320dd6fd8469 for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,095 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/92a9fcb53ed146f58cd20ee6828e019a for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,096 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/9974ec9e406648e58660186bb1d7f420 for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,097 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/aaf025f0f70e4a3e839770e623098e97 for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,097 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/ca9fd58be3704739a1150ab8f4f79ea4 for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,097 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/1a39c4a25ff8458590b3f608aa05825a for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,097 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/60125f034d6a4b318f6a5e94a304c87c for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,100 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/6bc92d7db7c04f2092d0d357544803c0 for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,100 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/6e24b414c2ba47f1bd35e83f1e758e6d for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,100 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/768ce0ae783f4044a3d4d153d4cfc4b9 for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,100 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/b24f5d9989ca4bb7945b363f3bc2a65a for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,109 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7058f4e7029c4e35912b425b094d4fb4 2023-02-09 14:54:30,109 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/aaf025f0f70e4a3e839770e623098e97, top=false 2023-02-09 14:54:30,114 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3259f9dfc059461f86903f033c01ca31 2023-02-09 14:54:30,114 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/6c84bb9704ec4bfcbadfb6d74ba97d77, top=false 2023-02-09 14:54:30,114 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/803c92c516a94497bd8e320dd6fd8469, top=false 2023-02-09 14:54:30,115 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 92a9fcb53ed146f58cd20ee6828e019a 2023-02-09 14:54:30,115 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6bc92d7db7c04f2092d0d357544803c0 2023-02-09 14:54:30,117 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ca9fd58be3704739a1150ab8f4f79ea4 2023-02-09 14:54:30,118 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1a39c4a25ff8458590b3f608aa05825a 2023-02-09 14:54:30,121 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/768ce0ae783f4044a3d4d153d4cfc4b9, top=false 2023-02-09 14:54:30,124 INFO [StoreFileSplitter-pool-4] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7058f4e7029c4e35912b425b094d4fb4 2023-02-09 14:54:30,124 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/7058f4e7029c4e35912b425b094d4fb4, top=true 2023-02-09 14:54:30,125 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 92a9fcb53ed146f58cd20ee6828e019a 2023-02-09 14:54:30,125 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/92a9fcb53ed146f58cd20ee6828e019a, top=true 2023-02-09 14:54:30,142 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3259f9dfc059461f86903f033c01ca31 2023-02-09 14:54:30,142 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/3259f9dfc059461f86903f033c01ca31, top=true 2023-02-09 14:54:30,142 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6bc92d7db7c04f2092d0d357544803c0 2023-02-09 14:54:30,142 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/6bc92d7db7c04f2092d0d357544803c0, top=true 2023-02-09 14:54:30,160 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.splits/c720e43e6c74641ba50283474b45a523/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-aaf025f0f70e4a3e839770e623098e97 for child: c720e43e6c74641ba50283474b45a523, parent: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,164 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/aaf025f0f70e4a3e839770e623098e97 for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,165 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/b2da05fec39f4cfd98dc4b8debb7657e for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,167 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/b2da05fec39f4cfd98dc4b8debb7657e, top=false 2023-02-09 14:54:30,170 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.splits/c720e43e6c74641ba50283474b45a523/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-803c92c516a94497bd8e320dd6fd8469 for child: c720e43e6c74641ba50283474b45a523, parent: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,171 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.splits/8c23d3f07541daba404c2b3d3332d0cb/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-92a9fcb53ed146f58cd20ee6828e019a for child: 8c23d3f07541daba404c2b3d3332d0cb, parent: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,171 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/92a9fcb53ed146f58cd20ee6828e019a for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,171 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/bec1a1460d3648ad83b9b6db4ce4809e for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,172 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.splits/c720e43e6c74641ba50283474b45a523/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-b2da05fec39f4cfd98dc4b8debb7657e for child: c720e43e6c74641ba50283474b45a523, parent: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,174 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/803c92c516a94497bd8e320dd6fd8469 for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,174 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/d1e0c6c3f72940668e505a5227f8b1bb for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,174 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/bec1a1460d3648ad83b9b6db4ce4809e, top=false 2023-02-09 14:54:30,175 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/b2da05fec39f4cfd98dc4b8debb7657e for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,175 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/f267f4025fb34ba3ab5ddd61ba69810b for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,178 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d1e0c6c3f72940668e505a5227f8b1bb 2023-02-09 14:54:30,179 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f267f4025fb34ba3ab5ddd61ba69810b 2023-02-09 14:54:30,180 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.splits/c720e43e6c74641ba50283474b45a523/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-bec1a1460d3648ad83b9b6db4ce4809e for child: c720e43e6c74641ba50283474b45a523, parent: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,182 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f267f4025fb34ba3ab5ddd61ba69810b 2023-02-09 14:54:30,182 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/f267f4025fb34ba3ab5ddd61ba69810b, top=true 2023-02-09 14:54:30,182 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.splits/8c23d3f07541daba404c2b3d3332d0cb/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-3259f9dfc059461f86903f033c01ca31 for child: 8c23d3f07541daba404c2b3d3332d0cb, parent: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,182 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/3259f9dfc059461f86903f033c01ca31 for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,183 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d1e0c6c3f72940668e505a5227f8b1bb 2023-02-09 14:54:30,184 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/d1e0c6c3f72940668e505a5227f8b1bb, top=true 2023-02-09 14:54:30,184 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.splits/c720e43e6c74641ba50283474b45a523/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-6c84bb9704ec4bfcbadfb6d74ba97d77 for child: c720e43e6c74641ba50283474b45a523, parent: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,184 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.splits/8c23d3f07541daba404c2b3d3332d0cb/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-6bc92d7db7c04f2092d0d357544803c0 for child: 8c23d3f07541daba404c2b3d3332d0cb, parent: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,185 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/6bc92d7db7c04f2092d0d357544803c0 for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,186 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/bec1a1460d3648ad83b9b6db4ce4809e for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,189 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/6c84bb9704ec4bfcbadfb6d74ba97d77 for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,193 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.splits/c720e43e6c74641ba50283474b45a523/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-768ce0ae783f4044a3d4d153d4cfc4b9 for child: c720e43e6c74641ba50283474b45a523, parent: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,199 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/768ce0ae783f4044a3d4d153d4cfc4b9 for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,211 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.splits/8c23d3f07541daba404c2b3d3332d0cb/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-7058f4e7029c4e35912b425b094d4fb4 for child: 8c23d3f07541daba404c2b3d3332d0cb, parent: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,211 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/7058f4e7029c4e35912b425b094d4fb4 for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,214 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ca9fd58be3704739a1150ab8f4f79ea4 2023-02-09 14:54:30,224 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.splits/8c23d3f07541daba404c2b3d3332d0cb/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-f267f4025fb34ba3ab5ddd61ba69810b for child: 8c23d3f07541daba404c2b3d3332d0cb, parent: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,224 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/f267f4025fb34ba3ab5ddd61ba69810b for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,224 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/.splits/8c23d3f07541daba404c2b3d3332d0cb/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-d1e0c6c3f72940668e505a5227f8b1bb for child: 8c23d3f07541daba404c2b3d3332d0cb, parent: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,225 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/d1e0c6c3f72940668e505a5227f8b1bb for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,234 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1a39c4a25ff8458590b3f608aa05825a 2023-02-09 14:54:30,246 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/39a737174efa406f8685bd11869876a4 for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,247 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/b24f5d9989ca4bb7945b363f3bc2a65a for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,250 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/9974ec9e406648e58660186bb1d7f420 for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,252 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/6e24b414c2ba47f1bd35e83f1e758e6d for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,253 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/6adf8d085dcc438aba69554d90273646 for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,257 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/ca9fd58be3704739a1150ab8f4f79ea4 for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,259 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/60125f034d6a4b318f6a5e94a304c87c for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,266 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/1a39c4a25ff8458590b3f608aa05825a for region: 94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,266 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region 94ffc82cbf109e7f71870e19837a0e78 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 14:54:30,302 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-09 14:54:30,304 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-09 14:54:30,306 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1675954470305"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1675954470305"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1675954470305"}]},"ts":"1675954470305"} 2023-02-09 14:54:30,306 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675954470305"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954470305"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954470305"}]},"ts":"1675954470305"} 2023-02-09 14:54:30,306 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675954470305"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954470305"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954470305"}]},"ts":"1675954470305"} 2023-02-09 14:54:30,309 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c720e43e6c74641ba50283474b45a523, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8c23d3f07541daba404c2b3d3332d0cb, ASSIGN}] 2023-02-09 14:54:30,309 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8c23d3f07541daba404c2b3d3332d0cb, ASSIGN 2023-02-09 14:54:30,310 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8c23d3f07541daba404c2b3d3332d0cb, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41077,1675954443212; forceNewPlan=false, retain=false 2023-02-09 14:54:30,310 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c720e43e6c74641ba50283474b45a523, ASSIGN 2023-02-09 14:54:30,311 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c720e43e6c74641ba50283474b45a523, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,41077,1675954443212; forceNewPlan=false, retain=false 2023-02-09 14:54:30,346 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 14:54:30,460 INFO [jenkins-hbase4:43829] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 14:54:30,462 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=8c23d3f07541daba404c2b3d3332d0cb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:30,462 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675954470462"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954470462"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954470462"}]},"ts":"1675954470462"} 2023-02-09 14:54:30,464 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; OpenRegionProcedure 8c23d3f07541daba404c2b3d3332d0cb, server=jenkins-hbase4.apache.org,41077,1675954443212}] 2023-02-09 14:54:30,466 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=c720e43e6c74641ba50283474b45a523, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:30,466 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675954470466"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954470466"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954470466"}]},"ts":"1675954470466"} 2023-02-09 14:54:30,467 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=75, state=RUNNABLE; OpenRegionProcedure c720e43e6c74641ba50283474b45a523, server=jenkins-hbase4.apache.org,41077,1675954443212}] 2023-02-09 14:54:30,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb. 2023-02-09 14:54:30,618 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8c23d3f07541daba404c2b3d3332d0cb, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.', STARTKEY => '60', ENDKEY => ''} 2023-02-09 14:54:30,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:54:30,618 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:54:30,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:30,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:54:30,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:54:30,620 INFO [StoreOpener-8c23d3f07541daba404c2b3d3332d0cb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:54:30,621 DEBUG [StoreOpener-8c23d3f07541daba404c2b3d3332d0cb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1 2023-02-09 14:54:30,621 DEBUG [StoreOpener-8c23d3f07541daba404c2b3d3332d0cb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1 2023-02-09 14:54:30,621 INFO [StoreOpener-8c23d3f07541daba404c2b3d3332d0cb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8c23d3f07541daba404c2b3d3332d0cb columnFamilyName cf1 2023-02-09 14:54:30,629 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1a39c4a25ff8458590b3f608aa05825a.94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,631 DEBUG [StoreOpener-8c23d3f07541daba404c2b3d3332d0cb-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/1a39c4a25ff8458590b3f608aa05825a.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/1a39c4a25ff8458590b3f608aa05825a-top 2023-02-09 14:54:30,635 DEBUG [StoreOpener-8c23d3f07541daba404c2b3d3332d0cb-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/60125f034d6a4b318f6a5e94a304c87c.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/60125f034d6a4b318f6a5e94a304c87c-top 2023-02-09 14:54:30,638 DEBUG [StoreOpener-8c23d3f07541daba404c2b3d3332d0cb-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/6e24b414c2ba47f1bd35e83f1e758e6d.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/6e24b414c2ba47f1bd35e83f1e758e6d-top 2023-02-09 14:54:30,641 DEBUG [StoreOpener-8c23d3f07541daba404c2b3d3332d0cb-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/b24f5d9989ca4bb7945b363f3bc2a65a.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/b24f5d9989ca4bb7945b363f3bc2a65a-top 2023-02-09 14:54:30,644 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6bc92d7db7c04f2092d0d357544803c0 2023-02-09 14:54:30,644 DEBUG [StoreOpener-8c23d3f07541daba404c2b3d3332d0cb-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-6bc92d7db7c04f2092d0d357544803c0 2023-02-09 14:54:30,647 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d1e0c6c3f72940668e505a5227f8b1bb 2023-02-09 14:54:30,647 DEBUG [StoreOpener-8c23d3f07541daba404c2b3d3332d0cb-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-d1e0c6c3f72940668e505a5227f8b1bb 2023-02-09 14:54:30,650 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f267f4025fb34ba3ab5ddd61ba69810b 2023-02-09 14:54:30,650 DEBUG [StoreOpener-8c23d3f07541daba404c2b3d3332d0cb-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-f267f4025fb34ba3ab5ddd61ba69810b 2023-02-09 14:54:30,650 INFO [StoreOpener-8c23d3f07541daba404c2b3d3332d0cb-1] regionserver.HStore(310): Store=8c23d3f07541daba404c2b3d3332d0cb/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:30,650 INFO [StoreOpener-8c23d3f07541daba404c2b3d3332d0cb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:54:30,651 DEBUG [StoreOpener-8c23d3f07541daba404c2b3d3332d0cb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2 2023-02-09 14:54:30,651 DEBUG [StoreOpener-8c23d3f07541daba404c2b3d3332d0cb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2 2023-02-09 14:54:30,651 INFO [StoreOpener-8c23d3f07541daba404c2b3d3332d0cb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8c23d3f07541daba404c2b3d3332d0cb columnFamilyName cf2 2023-02-09 14:54:30,660 DEBUG [StoreOpener-8c23d3f07541daba404c2b3d3332d0cb-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/39a737174efa406f8685bd11869876a4.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/39a737174efa406f8685bd11869876a4-top 2023-02-09 14:54:30,663 DEBUG [StoreOpener-8c23d3f07541daba404c2b3d3332d0cb-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/6adf8d085dcc438aba69554d90273646.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/6adf8d085dcc438aba69554d90273646-top 2023-02-09 14:54:30,666 DEBUG [StoreOpener-8c23d3f07541daba404c2b3d3332d0cb-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/9974ec9e406648e58660186bb1d7f420.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/9974ec9e406648e58660186bb1d7f420-top 2023-02-09 14:54:30,669 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ca9fd58be3704739a1150ab8f4f79ea4.94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,670 DEBUG [StoreOpener-8c23d3f07541daba404c2b3d3332d0cb-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/ca9fd58be3704739a1150ab8f4f79ea4.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/ca9fd58be3704739a1150ab8f4f79ea4-top 2023-02-09 14:54:30,672 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3259f9dfc059461f86903f033c01ca31 2023-02-09 14:54:30,672 DEBUG [StoreOpener-8c23d3f07541daba404c2b3d3332d0cb-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-3259f9dfc059461f86903f033c01ca31 2023-02-09 14:54:30,675 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7058f4e7029c4e35912b425b094d4fb4 2023-02-09 14:54:30,675 DEBUG [StoreOpener-8c23d3f07541daba404c2b3d3332d0cb-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-7058f4e7029c4e35912b425b094d4fb4 2023-02-09 14:54:30,677 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 92a9fcb53ed146f58cd20ee6828e019a 2023-02-09 14:54:30,677 DEBUG [StoreOpener-8c23d3f07541daba404c2b3d3332d0cb-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-92a9fcb53ed146f58cd20ee6828e019a 2023-02-09 14:54:30,678 INFO [StoreOpener-8c23d3f07541daba404c2b3d3332d0cb-1] regionserver.HStore(310): Store=8c23d3f07541daba404c2b3d3332d0cb/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:30,678 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:54:30,679 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:54:30,680 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:54:30,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:54:30,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 8c23d3f07541daba404c2b3d3332d0cb; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64602353, jitterRate=-0.03734992444515228}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:54:30,682 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 8c23d3f07541daba404c2b3d3332d0cb: Running coprocessor pre-open hook at 1675954470619Writing region info on filesystem at 1675954470619Initializing all the Stores at 1675954470619Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954470619Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954470620 (+1 ms)Cleaning up temporary data from old regions at 1675954470680 (+60 ms)Cleaning up detritus from prior splits at 1675954470680Running coprocessor post-open hooks at 1675954470682 (+2 ms)Region opened successfully at 1675954470682 2023-02-09 14:54:30,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb., pid=77, masterSystemTime=1675954470615 2023-02-09 14:54:30,684 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:30,685 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 14:54:30,685 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 14:54:30,685 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 14:54:30,685 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb. 2023-02-09 14:54:30,686 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStore(1912): 8c23d3f07541daba404c2b3d3332d0cb/cf1 is initiating minor compaction (all files) 2023-02-09 14:54:30,686 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb. 2023-02-09 14:54:30,686 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStore(1912): 8c23d3f07541daba404c2b3d3332d0cb/cf2 is initiating minor compaction (all files) 2023-02-09 14:54:30,686 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 8c23d3f07541daba404c2b3d3332d0cb/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb. 2023-02-09 14:54:30,686 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 8c23d3f07541daba404c2b3d3332d0cb/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb. 2023-02-09 14:54:30,686 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/6e24b414c2ba47f1bd35e83f1e758e6d.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/6e24b414c2ba47f1bd35e83f1e758e6d-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/60125f034d6a4b318f6a5e94a304c87c.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/60125f034d6a4b318f6a5e94a304c87c-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/b24f5d9989ca4bb7945b363f3bc2a65a.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/b24f5d9989ca4bb7945b363f3bc2a65a-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/1a39c4a25ff8458590b3f608aa05825a.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/1a39c4a25ff8458590b3f608aa05825a-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-d1e0c6c3f72940668e505a5227f8b1bb, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-f267f4025fb34ba3ab5ddd61ba69810b, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-6bc92d7db7c04f2092d0d357544803c0] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/.tmp, totalSize=34.8 K 2023-02-09 14:54:30,686 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/6adf8d085dcc438aba69554d90273646.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/6adf8d085dcc438aba69554d90273646-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/9974ec9e406648e58660186bb1d7f420.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/9974ec9e406648e58660186bb1d7f420-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/39a737174efa406f8685bd11869876a4.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/39a737174efa406f8685bd11869876a4-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/ca9fd58be3704739a1150ab8f4f79ea4.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/ca9fd58be3704739a1150ab8f4f79ea4-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-92a9fcb53ed146f58cd20ee6828e019a, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-7058f4e7029c4e35912b425b094d4fb4, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-3259f9dfc059461f86903f033c01ca31] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/.tmp, totalSize=34.8 K 2023-02-09 14:54:30,686 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 6e24b414c2ba47f1bd35e83f1e758e6d.94ffc82cbf109e7f71870e19837a0e78, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675954468195 2023-02-09 14:54:30,686 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting 6adf8d085dcc438aba69554d90273646.94ffc82cbf109e7f71870e19837a0e78, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675954468195 2023-02-09 14:54:30,687 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb. 2023-02-09 14:54:30,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb. 2023-02-09 14:54:30,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523. 2023-02-09 14:54:30,687 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 60125f034d6a4b318f6a5e94a304c87c.94ffc82cbf109e7f71870e19837a0e78, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675954468407 2023-02-09 14:54:30,687 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c720e43e6c74641ba50283474b45a523, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.', STARTKEY => '', ENDKEY => '60'} 2023-02-09 14:54:30,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:54:30,688 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter c720e43e6c74641ba50283474b45a523 2023-02-09 14:54:30,688 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:30,688 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for c720e43e6c74641ba50283474b45a523 2023-02-09 14:54:30,688 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for c720e43e6c74641ba50283474b45a523 2023-02-09 14:54:30,688 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting 9974ec9e406648e58660186bb1d7f420.94ffc82cbf109e7f71870e19837a0e78, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675954468407 2023-02-09 14:54:30,688 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting b24f5d9989ca4bb7945b363f3bc2a65a.94ffc82cbf109e7f71870e19837a0e78, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675954468593 2023-02-09 14:54:30,689 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=8c23d3f07541daba404c2b3d3332d0cb, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:30,689 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675954470688"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954470688"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954470688"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954470688"}]},"ts":"1675954470688"} 2023-02-09 14:54:30,689 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 1a39c4a25ff8458590b3f608aa05825a.94ffc82cbf109e7f71870e19837a0e78, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1675954469317 2023-02-09 14:54:30,689 INFO [StoreOpener-c720e43e6c74641ba50283474b45a523-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c720e43e6c74641ba50283474b45a523 2023-02-09 14:54:30,690 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting 39a737174efa406f8685bd11869876a4.94ffc82cbf109e7f71870e19837a0e78, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675954468593 2023-02-09 14:54:30,690 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-d1e0c6c3f72940668e505a5227f8b1bb, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-09 14:54:30,690 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting ca9fd58be3704739a1150ab8f4f79ea4.94ffc82cbf109e7f71870e19837a0e78, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1675954469317 2023-02-09 14:54:30,690 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-f267f4025fb34ba3ab5ddd61ba69810b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-09 14:54:30,691 DEBUG [StoreOpener-c720e43e6c74641ba50283474b45a523-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1 2023-02-09 14:54:30,691 DEBUG [StoreOpener-c720e43e6c74641ba50283474b45a523-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1 2023-02-09 14:54:30,691 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-92a9fcb53ed146f58cd20ee6828e019a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-09 14:54:30,691 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-6bc92d7db7c04f2092d0d357544803c0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-09 14:54:30,691 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-7058f4e7029c4e35912b425b094d4fb4, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-09 14:54:30,691 INFO [StoreOpener-c720e43e6c74641ba50283474b45a523-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c720e43e6c74641ba50283474b45a523 columnFamilyName cf1 2023-02-09 14:54:30,692 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-3259f9dfc059461f86903f033c01ca31, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-09 14:54:30,693 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-02-09 14:54:30,693 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; OpenRegionProcedure 8c23d3f07541daba404c2b3d3332d0cb, server=jenkins-hbase4.apache.org,41077,1675954443212 in 226 msec 2023-02-09 14:54:30,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8c23d3f07541daba404c2b3d3332d0cb, ASSIGN in 384 msec 2023-02-09 14:54:30,710 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1a39c4a25ff8458590b3f608aa05825a.94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,710 DEBUG [StoreOpener-c720e43e6c74641ba50283474b45a523-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/1a39c4a25ff8458590b3f608aa05825a.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/1a39c4a25ff8458590b3f608aa05825a-bottom 2023-02-09 14:54:30,713 DEBUG [StoreOpener-c720e43e6c74641ba50283474b45a523-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/60125f034d6a4b318f6a5e94a304c87c.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/60125f034d6a4b318f6a5e94a304c87c-bottom 2023-02-09 14:54:30,718 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8c23d3f07541daba404c2b3d3332d0cb#cf1#compaction#72 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:54:30,719 DEBUG [StoreOpener-c720e43e6c74641ba50283474b45a523-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/6e24b414c2ba47f1bd35e83f1e758e6d.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/6e24b414c2ba47f1bd35e83f1e758e6d-bottom 2023-02-09 14:54:30,719 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] throttle.PressureAwareThroughputController(145): 8c23d3f07541daba404c2b3d3332d0cb#cf2#compaction#73 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:54:30,723 DEBUG [StoreOpener-c720e43e6c74641ba50283474b45a523-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/b24f5d9989ca4bb7945b363f3bc2a65a.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/b24f5d9989ca4bb7945b363f3bc2a65a-bottom 2023-02-09 14:54:30,726 DEBUG [StoreOpener-c720e43e6c74641ba50283474b45a523-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-768ce0ae783f4044a3d4d153d4cfc4b9 2023-02-09 14:54:30,729 DEBUG [StoreOpener-c720e43e6c74641ba50283474b45a523-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-b2da05fec39f4cfd98dc4b8debb7657e 2023-02-09 14:54:30,731 DEBUG [StoreOpener-c720e43e6c74641ba50283474b45a523-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-bec1a1460d3648ad83b9b6db4ce4809e 2023-02-09 14:54:30,735 INFO [StoreOpener-c720e43e6c74641ba50283474b45a523-1] regionserver.HStore(310): Store=c720e43e6c74641ba50283474b45a523/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:30,735 INFO [StoreOpener-c720e43e6c74641ba50283474b45a523-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c720e43e6c74641ba50283474b45a523 2023-02-09 14:54:30,736 DEBUG [StoreOpener-c720e43e6c74641ba50283474b45a523-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2 2023-02-09 14:54:30,736 DEBUG [StoreOpener-c720e43e6c74641ba50283474b45a523-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2 2023-02-09 14:54:30,737 INFO [StoreOpener-c720e43e6c74641ba50283474b45a523-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c720e43e6c74641ba50283474b45a523 columnFamilyName cf2 2023-02-09 14:54:30,739 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 36ceda19b94040bb8be515a3b4dfe9b2: NONE, but ROW specified in column family configuration 2023-02-09 14:54:30,740 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/.tmp/cf1/36ceda19b94040bb8be515a3b4dfe9b2 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/36ceda19b94040bb8be515a3b4dfe9b2 2023-02-09 14:54:30,742 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 4eab88d09b1a401eaa68cc650285fee8: NONE, but ROW specified in column family configuration 2023-02-09 14:54:30,743 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/.tmp/cf2/4eab88d09b1a401eaa68cc650285fee8 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/4eab88d09b1a401eaa68cc650285fee8 2023-02-09 14:54:30,745 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 36ceda19b94040bb8be515a3b4dfe9b2: NONE, but ROW specified in column family configuration 2023-02-09 14:54:30,746 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 8c23d3f07541daba404c2b3d3332d0cb/cf1 of 8c23d3f07541daba404c2b3d3332d0cb into 36ceda19b94040bb8be515a3b4dfe9b2(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:54:30,746 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 8c23d3f07541daba404c2b3d3332d0cb: 2023-02-09 14:54:30,746 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb., storeName=8c23d3f07541daba404c2b3d3332d0cb/cf1, priority=9, startTime=1675954470684; duration=0sec 2023-02-09 14:54:30,746 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:30,747 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 4eab88d09b1a401eaa68cc650285fee8: NONE, but ROW specified in column family configuration 2023-02-09 14:54:30,748 DEBUG [StoreOpener-c720e43e6c74641ba50283474b45a523-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/39a737174efa406f8685bd11869876a4.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/39a737174efa406f8685bd11869876a4-bottom 2023-02-09 14:54:30,748 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 8c23d3f07541daba404c2b3d3332d0cb/cf2 of 8c23d3f07541daba404c2b3d3332d0cb into 4eab88d09b1a401eaa68cc650285fee8(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:54:30,748 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 8c23d3f07541daba404c2b3d3332d0cb: 2023-02-09 14:54:30,748 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb., storeName=8c23d3f07541daba404c2b3d3332d0cb/cf2, priority=9, startTime=1675954470684; duration=0sec 2023-02-09 14:54:30,748 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:30,751 DEBUG [StoreOpener-c720e43e6c74641ba50283474b45a523-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/6adf8d085dcc438aba69554d90273646.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/6adf8d085dcc438aba69554d90273646-bottom 2023-02-09 14:54:30,754 DEBUG [StoreOpener-c720e43e6c74641ba50283474b45a523-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/9974ec9e406648e58660186bb1d7f420.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/9974ec9e406648e58660186bb1d7f420-bottom 2023-02-09 14:54:30,757 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ca9fd58be3704739a1150ab8f4f79ea4.94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:30,757 DEBUG [StoreOpener-c720e43e6c74641ba50283474b45a523-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/ca9fd58be3704739a1150ab8f4f79ea4.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/ca9fd58be3704739a1150ab8f4f79ea4-bottom 2023-02-09 14:54:30,760 DEBUG [StoreOpener-c720e43e6c74641ba50283474b45a523-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-6c84bb9704ec4bfcbadfb6d74ba97d77 2023-02-09 14:54:30,763 DEBUG [StoreOpener-c720e43e6c74641ba50283474b45a523-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-803c92c516a94497bd8e320dd6fd8469 2023-02-09 14:54:30,767 DEBUG [StoreOpener-c720e43e6c74641ba50283474b45a523-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-aaf025f0f70e4a3e839770e623098e97 2023-02-09 14:54:30,767 INFO [StoreOpener-c720e43e6c74641ba50283474b45a523-1] regionserver.HStore(310): Store=c720e43e6c74641ba50283474b45a523/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:30,768 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523 2023-02-09 14:54:30,768 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523 2023-02-09 14:54:30,770 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:54:30,771 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for c720e43e6c74641ba50283474b45a523 2023-02-09 14:54:30,771 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened c720e43e6c74641ba50283474b45a523; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74427277, jitterRate=0.10905285179615021}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:54:30,771 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for c720e43e6c74641ba50283474b45a523: Running coprocessor pre-open hook at 1675954470688Writing region info on filesystem at 1675954470688Initializing all the Stores at 1675954470689 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954470689Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954470689Cleaning up temporary data from old regions at 1675954470769 (+80 ms)Cleaning up detritus from prior splits at 1675954470769Running coprocessor post-open hooks at 1675954470771 (+2 ms)Region opened successfully at 1675954470771 2023-02-09 14:54:30,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523., pid=78, masterSystemTime=1675954470615 2023-02-09 14:54:30,772 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:30,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:30,773 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 14:54:30,773 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 14:54:30,773 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523. 2023-02-09 14:54:30,773 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523. 2023-02-09 14:54:30,773 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStore(1912): c720e43e6c74641ba50283474b45a523/cf2 is initiating minor compaction (all files) 2023-02-09 14:54:30,773 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStore(1912): c720e43e6c74641ba50283474b45a523/cf1 is initiating minor compaction (all files) 2023-02-09 14:54:30,773 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HRegion(2258): Starting compaction of c720e43e6c74641ba50283474b45a523/cf2 in testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523. 2023-02-09 14:54:30,773 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of c720e43e6c74641ba50283474b45a523/cf1 in testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523. 2023-02-09 14:54:30,774 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/6adf8d085dcc438aba69554d90273646.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/6adf8d085dcc438aba69554d90273646-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/9974ec9e406648e58660186bb1d7f420.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/9974ec9e406648e58660186bb1d7f420-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/39a737174efa406f8685bd11869876a4.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/39a737174efa406f8685bd11869876a4-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-aaf025f0f70e4a3e839770e623098e97, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-803c92c516a94497bd8e320dd6fd8469, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-6c84bb9704ec4bfcbadfb6d74ba97d77, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/ca9fd58be3704739a1150ab8f4f79ea4.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/ca9fd58be3704739a1150ab8f4f79ea4-bottom] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/.tmp, totalSize=35.0 K 2023-02-09 14:54:30,774 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/6e24b414c2ba47f1bd35e83f1e758e6d.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/6e24b414c2ba47f1bd35e83f1e758e6d-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/60125f034d6a4b318f6a5e94a304c87c.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/60125f034d6a4b318f6a5e94a304c87c-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/b24f5d9989ca4bb7945b363f3bc2a65a.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/b24f5d9989ca4bb7945b363f3bc2a65a-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-bec1a1460d3648ad83b9b6db4ce4809e, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-b2da05fec39f4cfd98dc4b8debb7657e, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-768ce0ae783f4044a3d4d153d4cfc4b9, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/1a39c4a25ff8458590b3f608aa05825a.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/1a39c4a25ff8458590b3f608aa05825a-bottom] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/.tmp, totalSize=35.0 K 2023-02-09 14:54:30,774 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting 6adf8d085dcc438aba69554d90273646.94ffc82cbf109e7f71870e19837a0e78, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675954468195 2023-02-09 14:54:30,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523. 2023-02-09 14:54:30,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523. 2023-02-09 14:54:30,775 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 6e24b414c2ba47f1bd35e83f1e758e6d.94ffc82cbf109e7f71870e19837a0e78, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675954468195 2023-02-09 14:54:30,775 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=c720e43e6c74641ba50283474b45a523, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:30,775 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting 9974ec9e406648e58660186bb1d7f420.94ffc82cbf109e7f71870e19837a0e78, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675954468407 2023-02-09 14:54:30,775 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 60125f034d6a4b318f6a5e94a304c87c.94ffc82cbf109e7f71870e19837a0e78, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675954468407 2023-02-09 14:54:30,775 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675954470775"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954470775"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954470775"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954470775"}]},"ts":"1675954470775"} 2023-02-09 14:54:30,775 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting 39a737174efa406f8685bd11869876a4.94ffc82cbf109e7f71870e19837a0e78, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675954468593 2023-02-09 14:54:30,776 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting b24f5d9989ca4bb7945b363f3bc2a65a.94ffc82cbf109e7f71870e19837a0e78, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675954468593 2023-02-09 14:54:30,776 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-aaf025f0f70e4a3e839770e623098e97, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675954468780 2023-02-09 14:54:30,776 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-bec1a1460d3648ad83b9b6db4ce4809e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675954468780 2023-02-09 14:54:30,776 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-803c92c516a94497bd8e320dd6fd8469, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675954468962 2023-02-09 14:54:30,776 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-b2da05fec39f4cfd98dc4b8debb7657e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675954468962 2023-02-09 14:54:30,777 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-6c84bb9704ec4bfcbadfb6d74ba97d77, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675954469136 2023-02-09 14:54:30,777 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-768ce0ae783f4044a3d4d153d4cfc4b9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675954469136 2023-02-09 14:54:30,777 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting ca9fd58be3704739a1150ab8f4f79ea4.94ffc82cbf109e7f71870e19837a0e78, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675954469317 2023-02-09 14:54:30,777 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 1a39c4a25ff8458590b3f608aa05825a.94ffc82cbf109e7f71870e19837a0e78, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675954469317 2023-02-09 14:54:30,779 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=75 2023-02-09 14:54:30,779 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=75, state=SUCCESS; OpenRegionProcedure c720e43e6c74641ba50283474b45a523, server=jenkins-hbase4.apache.org,41077,1675954443212 in 309 msec 2023-02-09 14:54:30,781 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=75, resume processing ppid=72 2023-02-09 14:54:30,781 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c720e43e6c74641ba50283474b45a523, ASSIGN in 470 msec 2023-02-09 14:54:30,782 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=94ffc82cbf109e7f71870e19837a0e78, daughterA=c720e43e6c74641ba50283474b45a523, daughterB=8c23d3f07541daba404c2b3d3332d0cb in 908 msec 2023-02-09 14:54:30,791 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] throttle.PressureAwareThroughputController(145): c720e43e6c74641ba50283474b45a523#cf1#compaction#75 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:54:30,791 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] throttle.PressureAwareThroughputController(145): c720e43e6c74641ba50283474b45a523#cf2#compaction#74 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:54:30,807 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/.tmp/cf1/5f57d68cafd14b4682b90481bcf3b8a7 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/5f57d68cafd14b4682b90481bcf3b8a7 2023-02-09 14:54:30,807 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/.tmp/cf2/046e66af4090475a9bde0cdb13dee1a7 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/046e66af4090475a9bde0cdb13dee1a7 2023-02-09 14:54:30,812 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c720e43e6c74641ba50283474b45a523/cf1 of c720e43e6c74641ba50283474b45a523 into 5f57d68cafd14b4682b90481bcf3b8a7(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:54:30,812 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c720e43e6c74641ba50283474b45a523/cf2 of c720e43e6c74641ba50283474b45a523 into 046e66af4090475a9bde0cdb13dee1a7(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:54:30,812 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for c720e43e6c74641ba50283474b45a523: 2023-02-09 14:54:30,812 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for c720e43e6c74641ba50283474b45a523: 2023-02-09 14:54:30,812 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523., storeName=c720e43e6c74641ba50283474b45a523/cf1, priority=9, startTime=1675954470772; duration=0sec 2023-02-09 14:54:30,812 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523., storeName=c720e43e6c74641ba50283474b45a523/cf2, priority=9, startTime=1675954470772; duration=0sec 2023-02-09 14:54:30,812 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:30,812 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:30,884 DEBUG [Listener at localhost/40823] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 14:54:30,887 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52736, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 14:54:30,887 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41077] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523. 2023-02-09 14:54:30,888 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41077] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-09 14:54:30,888 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41077] regionserver.HStore(1912): c720e43e6c74641ba50283474b45a523/cf1 is initiating major compaction (all files) 2023-02-09 14:54:30,889 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41077] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@28e43d00; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:30,889 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of c720e43e6c74641ba50283474b45a523/cf1 in testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523. 2023-02-09 14:54:30,889 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41077] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-09 14:54:30,889 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/5f57d68cafd14b4682b90481bcf3b8a7] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/.tmp, totalSize=6.8 K 2023-02-09 14:54:30,889 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41077] regionserver.HStore(1912): c720e43e6c74641ba50283474b45a523/cf2 is initiating major compaction (all files) 2023-02-09 14:54:30,889 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41077] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4a6b2b36; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 14:54:30,889 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HRegion(2258): Starting compaction of c720e43e6c74641ba50283474b45a523/cf2 in testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523. 2023-02-09 14:54:30,889 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/046e66af4090475a9bde0cdb13dee1a7] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/.tmp, totalSize=6.8 K 2023-02-09 14:54:30,889 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 5f57d68cafd14b4682b90481bcf3b8a7, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675954468195 2023-02-09 14:54:30,889 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting 046e66af4090475a9bde0cdb13dee1a7, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675954468195 2023-02-09 14:54:30,891 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41077] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb. 2023-02-09 14:54:30,891 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41077] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-09 14:54:30,891 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41077] regionserver.HStore(1912): 8c23d3f07541daba404c2b3d3332d0cb/cf1 is initiating major compaction (all files) 2023-02-09 14:54:30,891 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41077] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@9eb6c38; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 14:54:30,891 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41077] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-09 14:54:30,891 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41077] regionserver.HStore(1912): 8c23d3f07541daba404c2b3d3332d0cb/cf2 is initiating major compaction (all files) 2023-02-09 14:54:30,891 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41077] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3d781207; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-09 14:54:30,892 INFO [Listener at localhost/40823] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-02-09 14:54:30,894 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] throttle.PressureAwareThroughputController(145): c720e43e6c74641ba50283474b45a523#cf1#compaction#76 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:54:30,894 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] throttle.PressureAwareThroughputController(145): c720e43e6c74641ba50283474b45a523#cf2#compaction#77 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:54:30,915 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41077] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523. is not mergeable because it has references 2023-02-09 14:54:30,916 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41077] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb. is not mergeable because it has references 2023-02-09 14:54:30,917 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/.tmp/cf1/bd5129517ef04b6e9387689148f22866 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/bd5129517ef04b6e9387689148f22866 2023-02-09 14:54:30,918 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/.tmp/cf2/f3af9907545f458b8a701e8e79306603 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/f3af9907545f458b8a701e8e79306603 2023-02-09 14:54:30,922 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in c720e43e6c74641ba50283474b45a523/cf1 of c720e43e6c74641ba50283474b45a523 into bd5129517ef04b6e9387689148f22866(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:54:30,922 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for c720e43e6c74641ba50283474b45a523: 2023-02-09 14:54:30,922 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in c720e43e6c74641ba50283474b45a523/cf2 of c720e43e6c74641ba50283474b45a523 into f3af9907545f458b8a701e8e79306603(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:54:30,922 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for c720e43e6c74641ba50283474b45a523: 2023-02-09 14:54:30,923 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1675954470888; duration=0sec 2023-02-09 14:54:30,923 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1675954470889; duration=0sec 2023-02-09 14:54:30,923 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-09 14:54:30,923 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 14:54:30,923 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 8c23d3f07541daba404c2b3d3332d0cb/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb. 2023-02-09 14:54:30,923 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 8c23d3f07541daba404c2b3d3332d0cb/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb. 2023-02-09 14:54:30,923 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/4eab88d09b1a401eaa68cc650285fee8] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/.tmp, totalSize=5.0 K 2023-02-09 14:54:30,924 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/36ceda19b94040bb8be515a3b4dfe9b2] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/.tmp, totalSize=5.0 K 2023-02-09 14:54:30,924 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] compactions.Compactor(207): Compacting 4eab88d09b1a401eaa68cc650285fee8, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-09 14:54:30,924 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] compactions.Compactor(207): Compacting 36ceda19b94040bb8be515a3b4dfe9b2, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-09 14:54:30,927 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8c23d3f07541daba404c2b3d3332d0cb#cf2#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:54:30,927 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] throttle.PressureAwareThroughputController(145): 8c23d3f07541daba404c2b3d3332d0cb#cf1#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:54:30,945 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 17e1e7f7c7c14bb890ae8c65bf81cee1: NONE, but ROW specified in column family configuration 2023-02-09 14:54:30,945 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 82181e8ba11545b486a6d1fbf4e6e3e4: NONE, but ROW specified in column family configuration 2023-02-09 14:54:30,946 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/.tmp/cf1/17e1e7f7c7c14bb890ae8c65bf81cee1 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/17e1e7f7c7c14bb890ae8c65bf81cee1 2023-02-09 14:54:30,952 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/.tmp/cf2/82181e8ba11545b486a6d1fbf4e6e3e4 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/82181e8ba11545b486a6d1fbf4e6e3e4 2023-02-09 14:54:30,959 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 17e1e7f7c7c14bb890ae8c65bf81cee1: NONE, but ROW specified in column family configuration 2023-02-09 14:54:30,960 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 82181e8ba11545b486a6d1fbf4e6e3e4: NONE, but ROW specified in column family configuration 2023-02-09 14:54:30,960 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 8c23d3f07541daba404c2b3d3332d0cb/cf1 of 8c23d3f07541daba404c2b3d3332d0cb into 17e1e7f7c7c14bb890ae8c65bf81cee1(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:54:30,960 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 8c23d3f07541daba404c2b3d3332d0cb: 2023-02-09 14:54:30,961 INFO [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1675954470891; duration=0sec 2023-02-09 14:54:30,961 DEBUG [RS:2;jenkins-hbase4:41077-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:30,961 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 8c23d3f07541daba404c2b3d3332d0cb/cf2 of 8c23d3f07541daba404c2b3d3332d0cb into 82181e8ba11545b486a6d1fbf4e6e3e4(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:54:30,961 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 8c23d3f07541daba404c2b3d3332d0cb: 2023-02-09 14:54:30,961 INFO [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1675954470891; duration=0sec 2023-02-09 14:54:30,962 DEBUG [RS:2;jenkins-hbase4:41077-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:54:31,026 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41077] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523. is not mergeable because it has references 2023-02-09 14:54:31,026 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41077] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb. is not mergeable because it has references 2023-02-09 14:54:31,032 DEBUG [Listener at localhost/40823] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-02-09 14:54:31,034 WARN [Listener at localhost/40823] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 14:54:31,034 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:31,034 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:31,035 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:31,035 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:31,035 INFO [Listener at localhost/40823] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:31,036 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:31,037 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:31,038 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:31,038 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:31,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 14:54:31,040 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954471040"}]},"ts":"1675954471040"} 2023-02-09 14:54:31,041 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-02-09 14:54:31,042 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-02-09 14:54:31,043 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c720e43e6c74641ba50283474b45a523, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8c23d3f07541daba404c2b3d3332d0cb, UNASSIGN}] 2023-02-09 14:54:31,044 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8c23d3f07541daba404c2b3d3332d0cb, UNASSIGN 2023-02-09 14:54:31,045 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=8c23d3f07541daba404c2b3d3332d0cb, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:31,045 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675954471045"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954471045"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954471045"}]},"ts":"1675954471045"} 2023-02-09 14:54:31,046 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure 8c23d3f07541daba404c2b3d3332d0cb, server=jenkins-hbase4.apache.org,41077,1675954443212}] 2023-02-09 14:54:31,048 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c720e43e6c74641ba50283474b45a523, UNASSIGN 2023-02-09 14:54:31,049 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=c720e43e6c74641ba50283474b45a523, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:31,049 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675954471049"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954471049"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954471049"}]},"ts":"1675954471049"} 2023-02-09 14:54:31,050 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure c720e43e6c74641ba50283474b45a523, server=jenkins-hbase4.apache.org,41077,1675954443212}] 2023-02-09 14:54:31,139 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:31,139 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:31,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 14:54:31,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:54:31,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 8c23d3f07541daba404c2b3d3332d0cb, disabling compactions & flushes 2023-02-09 14:54:31,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb. 2023-02-09 14:54:31,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb. 2023-02-09 14:54:31,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb. after waiting 0 ms 2023-02-09 14:54:31,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb. 2023-02-09 14:54:31,199 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/6e24b414c2ba47f1bd35e83f1e758e6d.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/6e24b414c2ba47f1bd35e83f1e758e6d-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/60125f034d6a4b318f6a5e94a304c87c.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/60125f034d6a4b318f6a5e94a304c87c-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/b24f5d9989ca4bb7945b363f3bc2a65a.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/b24f5d9989ca4bb7945b363f3bc2a65a-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/1a39c4a25ff8458590b3f608aa05825a.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/1a39c4a25ff8458590b3f608aa05825a-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-d1e0c6c3f72940668e505a5227f8b1bb, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-f267f4025fb34ba3ab5ddd61ba69810b, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/36ceda19b94040bb8be515a3b4dfe9b2, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-6bc92d7db7c04f2092d0d357544803c0] to archive 2023-02-09 14:54:31,200 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:31,201 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/6e24b414c2ba47f1bd35e83f1e758e6d.94ffc82cbf109e7f71870e19837a0e78 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/6e24b414c2ba47f1bd35e83f1e758e6d.94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:31,203 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/60125f034d6a4b318f6a5e94a304c87c.94ffc82cbf109e7f71870e19837a0e78 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/60125f034d6a4b318f6a5e94a304c87c.94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:31,204 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/b24f5d9989ca4bb7945b363f3bc2a65a.94ffc82cbf109e7f71870e19837a0e78 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/b24f5d9989ca4bb7945b363f3bc2a65a.94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:31,205 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/1a39c4a25ff8458590b3f608aa05825a.94ffc82cbf109e7f71870e19837a0e78 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/1a39c4a25ff8458590b3f608aa05825a.94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:31,206 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-d1e0c6c3f72940668e505a5227f8b1bb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-d1e0c6c3f72940668e505a5227f8b1bb 2023-02-09 14:54:31,207 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-f267f4025fb34ba3ab5ddd61ba69810b to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-f267f4025fb34ba3ab5ddd61ba69810b 2023-02-09 14:54:31,208 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/36ceda19b94040bb8be515a3b4dfe9b2 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/36ceda19b94040bb8be515a3b4dfe9b2 2023-02-09 14:54:31,209 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-6bc92d7db7c04f2092d0d357544803c0 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-6bc92d7db7c04f2092d0d357544803c0 2023-02-09 14:54:31,211 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/6adf8d085dcc438aba69554d90273646.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/6adf8d085dcc438aba69554d90273646-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/9974ec9e406648e58660186bb1d7f420.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/9974ec9e406648e58660186bb1d7f420-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/39a737174efa406f8685bd11869876a4.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/39a737174efa406f8685bd11869876a4-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/ca9fd58be3704739a1150ab8f4f79ea4.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/ca9fd58be3704739a1150ab8f4f79ea4-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-92a9fcb53ed146f58cd20ee6828e019a, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-7058f4e7029c4e35912b425b094d4fb4, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/4eab88d09b1a401eaa68cc650285fee8, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-3259f9dfc059461f86903f033c01ca31] to archive 2023-02-09 14:54:31,211 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:31,213 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/6adf8d085dcc438aba69554d90273646.94ffc82cbf109e7f71870e19837a0e78 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/6adf8d085dcc438aba69554d90273646.94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:31,214 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/9974ec9e406648e58660186bb1d7f420.94ffc82cbf109e7f71870e19837a0e78 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/9974ec9e406648e58660186bb1d7f420.94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:31,215 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/39a737174efa406f8685bd11869876a4.94ffc82cbf109e7f71870e19837a0e78 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/39a737174efa406f8685bd11869876a4.94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:31,216 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/ca9fd58be3704739a1150ab8f4f79ea4.94ffc82cbf109e7f71870e19837a0e78 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/ca9fd58be3704739a1150ab8f4f79ea4.94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:31,217 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-92a9fcb53ed146f58cd20ee6828e019a to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-92a9fcb53ed146f58cd20ee6828e019a 2023-02-09 14:54:31,217 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-7058f4e7029c4e35912b425b094d4fb4 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-7058f4e7029c4e35912b425b094d4fb4 2023-02-09 14:54:31,218 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/4eab88d09b1a401eaa68cc650285fee8 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/4eab88d09b1a401eaa68cc650285fee8 2023-02-09 14:54:31,219 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-3259f9dfc059461f86903f033c01ca31 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-3259f9dfc059461f86903f033c01ca31 2023-02-09 14:54:31,222 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-09 14:54:31,223 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:54:31,223 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb. 2023-02-09 14:54:31,223 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 8c23d3f07541daba404c2b3d3332d0cb: Waiting for close lock at 1675954471198Running coprocessor pre-close hooks at 1675954471198Disabling compacts and flushes for region at 1675954471198Disabling writes for close at 1675954471198Writing region close event to WAL at 1675954471220 (+22 ms)Running coprocessor post-close hooks at 1675954471223 (+3 ms)Closed at 1675954471223 2023-02-09 14:54:31,224 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:54:31,224 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close c720e43e6c74641ba50283474b45a523 2023-02-09 14:54:31,224 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing c720e43e6c74641ba50283474b45a523, disabling compactions & flushes 2023-02-09 14:54:31,224 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523. 2023-02-09 14:54:31,225 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523. 2023-02-09 14:54:31,225 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523. after waiting 0 ms 2023-02-09 14:54:31,225 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523. 2023-02-09 14:54:31,225 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=8c23d3f07541daba404c2b3d3332d0cb, regionState=CLOSED 2023-02-09 14:54:31,225 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675954471225"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954471225"}]},"ts":"1675954471225"} 2023-02-09 14:54:31,226 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/6e24b414c2ba47f1bd35e83f1e758e6d.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/6e24b414c2ba47f1bd35e83f1e758e6d-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/60125f034d6a4b318f6a5e94a304c87c.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/60125f034d6a4b318f6a5e94a304c87c-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/b24f5d9989ca4bb7945b363f3bc2a65a.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/b24f5d9989ca4bb7945b363f3bc2a65a-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-bec1a1460d3648ad83b9b6db4ce4809e, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-b2da05fec39f4cfd98dc4b8debb7657e, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-768ce0ae783f4044a3d4d153d4cfc4b9, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/5f57d68cafd14b4682b90481bcf3b8a7, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/1a39c4a25ff8458590b3f608aa05825a.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/1a39c4a25ff8458590b3f608aa05825a-bottom] to archive 2023-02-09 14:54:31,226 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:31,228 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/6e24b414c2ba47f1bd35e83f1e758e6d.94ffc82cbf109e7f71870e19837a0e78 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/6e24b414c2ba47f1bd35e83f1e758e6d.94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:31,228 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-02-09 14:54:31,228 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure 8c23d3f07541daba404c2b3d3332d0cb, server=jenkins-hbase4.apache.org,41077,1675954443212 in 181 msec 2023-02-09 14:54:31,229 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/60125f034d6a4b318f6a5e94a304c87c.94ffc82cbf109e7f71870e19837a0e78 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/60125f034d6a4b318f6a5e94a304c87c.94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:31,229 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=8c23d3f07541daba404c2b3d3332d0cb, UNASSIGN in 186 msec 2023-02-09 14:54:31,230 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/b24f5d9989ca4bb7945b363f3bc2a65a.94ffc82cbf109e7f71870e19837a0e78 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/b24f5d9989ca4bb7945b363f3bc2a65a.94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:31,231 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-bec1a1460d3648ad83b9b6db4ce4809e to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-bec1a1460d3648ad83b9b6db4ce4809e 2023-02-09 14:54:31,232 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-b2da05fec39f4cfd98dc4b8debb7657e to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-b2da05fec39f4cfd98dc4b8debb7657e 2023-02-09 14:54:31,233 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-768ce0ae783f4044a3d4d153d4cfc4b9 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-768ce0ae783f4044a3d4d153d4cfc4b9 2023-02-09 14:54:31,234 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/5f57d68cafd14b4682b90481bcf3b8a7 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/5f57d68cafd14b4682b90481bcf3b8a7 2023-02-09 14:54:31,235 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/1a39c4a25ff8458590b3f608aa05825a.94ffc82cbf109e7f71870e19837a0e78 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/1a39c4a25ff8458590b3f608aa05825a.94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:31,236 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/6adf8d085dcc438aba69554d90273646.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/6adf8d085dcc438aba69554d90273646-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/9974ec9e406648e58660186bb1d7f420.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/9974ec9e406648e58660186bb1d7f420-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/39a737174efa406f8685bd11869876a4.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/39a737174efa406f8685bd11869876a4-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-aaf025f0f70e4a3e839770e623098e97, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-803c92c516a94497bd8e320dd6fd8469, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-6c84bb9704ec4bfcbadfb6d74ba97d77, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/046e66af4090475a9bde0cdb13dee1a7, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/ca9fd58be3704739a1150ab8f4f79ea4.94ffc82cbf109e7f71870e19837a0e78->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/ca9fd58be3704739a1150ab8f4f79ea4-bottom] to archive 2023-02-09 14:54:31,237 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:54:31,238 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/6adf8d085dcc438aba69554d90273646.94ffc82cbf109e7f71870e19837a0e78 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/6adf8d085dcc438aba69554d90273646.94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:31,239 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/9974ec9e406648e58660186bb1d7f420.94ffc82cbf109e7f71870e19837a0e78 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/9974ec9e406648e58660186bb1d7f420.94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:31,240 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/39a737174efa406f8685bd11869876a4.94ffc82cbf109e7f71870e19837a0e78 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/39a737174efa406f8685bd11869876a4.94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:31,241 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-aaf025f0f70e4a3e839770e623098e97 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-aaf025f0f70e4a3e839770e623098e97 2023-02-09 14:54:31,242 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-803c92c516a94497bd8e320dd6fd8469 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-803c92c516a94497bd8e320dd6fd8469 2023-02-09 14:54:31,243 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-6c84bb9704ec4bfcbadfb6d74ba97d77 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/testSplitTableRegionDeletedRowsDaughter=94ffc82cbf109e7f71870e19837a0e78-6c84bb9704ec4bfcbadfb6d74ba97d77 2023-02-09 14:54:31,244 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/046e66af4090475a9bde0cdb13dee1a7 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/046e66af4090475a9bde0cdb13dee1a7 2023-02-09 14:54:31,245 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/ca9fd58be3704739a1150ab8f4f79ea4.94ffc82cbf109e7f71870e19837a0e78 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/ca9fd58be3704739a1150ab8f4f79ea4.94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:31,248 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-09 14:54:31,249 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:54:31,249 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523. 2023-02-09 14:54:31,249 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for c720e43e6c74641ba50283474b45a523: Waiting for close lock at 1675954471224Running coprocessor pre-close hooks at 1675954471224Disabling compacts and flushes for region at 1675954471224Disabling writes for close at 1675954471225 (+1 ms)Writing region close event to WAL at 1675954471246 (+21 ms)Running coprocessor post-close hooks at 1675954471249 (+3 ms)Closed at 1675954471249 2023-02-09 14:54:31,250 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed c720e43e6c74641ba50283474b45a523 2023-02-09 14:54:31,250 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=c720e43e6c74641ba50283474b45a523, regionState=CLOSED 2023-02-09 14:54:31,251 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1675954471250"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954471250"}]},"ts":"1675954471250"} 2023-02-09 14:54:31,253 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-09 14:54:31,253 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure c720e43e6c74641ba50283474b45a523, server=jenkins-hbase4.apache.org,41077,1675954443212 in 202 msec 2023-02-09 14:54:31,254 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-02-09 14:54:31,254 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=c720e43e6c74641ba50283474b45a523, UNASSIGN in 210 msec 2023-02-09 14:54:31,255 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954471255"}]},"ts":"1675954471255"} 2023-02-09 14:54:31,256 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-02-09 14:54:31,257 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-02-09 14:54:31,259 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 222 msec 2023-02-09 14:54:31,341 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:31,341 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:31,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-09 14:54:31,342 INFO [Listener at localhost/40823] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-02-09 14:54:31,342 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:31,342 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:31,343 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:31,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:31,345 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:31,345 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:31,345 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:31,346 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:31,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-09 14:54:31,350 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:31,350 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523 2023-02-09 14:54:31,350 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:54:31,352 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/recovered.edits] 2023-02-09 14:54:31,352 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/recovered.edits] 2023-02-09 14:54:31,352 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/recovered.edits] 2023-02-09 14:54:31,355 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/bd5129517ef04b6e9387689148f22866 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf1/bd5129517ef04b6e9387689148f22866 2023-02-09 14:54:31,355 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/17e1e7f7c7c14bb890ae8c65bf81cee1 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf1/17e1e7f7c7c14bb890ae8c65bf81cee1 2023-02-09 14:54:31,355 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/1a39c4a25ff8458590b3f608aa05825a to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/1a39c4a25ff8458590b3f608aa05825a 2023-02-09 14:54:31,356 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/60125f034d6a4b318f6a5e94a304c87c to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/60125f034d6a4b318f6a5e94a304c87c 2023-02-09 14:54:31,357 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/82181e8ba11545b486a6d1fbf4e6e3e4 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/cf2/82181e8ba11545b486a6d1fbf4e6e3e4 2023-02-09 14:54:31,357 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/f3af9907545f458b8a701e8e79306603 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/cf2/f3af9907545f458b8a701e8e79306603 2023-02-09 14:54:31,358 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/6bc92d7db7c04f2092d0d357544803c0 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/6bc92d7db7c04f2092d0d357544803c0 2023-02-09 14:54:31,359 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/6e24b414c2ba47f1bd35e83f1e758e6d to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/6e24b414c2ba47f1bd35e83f1e758e6d 2023-02-09 14:54:31,360 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/recovered.edits/113.seqid to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523/recovered.edits/113.seqid 2023-02-09 14:54:31,360 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/recovered.edits/113.seqid to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb/recovered.edits/113.seqid 2023-02-09 14:54:31,361 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/768ce0ae783f4044a3d4d153d4cfc4b9 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/768ce0ae783f4044a3d4d153d4cfc4b9 2023-02-09 14:54:31,361 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/c720e43e6c74641ba50283474b45a523 2023-02-09 14:54:31,361 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:54:31,362 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/b24f5d9989ca4bb7945b363f3bc2a65a to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/b24f5d9989ca4bb7945b363f3bc2a65a 2023-02-09 14:54:31,363 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/b2da05fec39f4cfd98dc4b8debb7657e to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/b2da05fec39f4cfd98dc4b8debb7657e 2023-02-09 14:54:31,364 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/bec1a1460d3648ad83b9b6db4ce4809e to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/bec1a1460d3648ad83b9b6db4ce4809e 2023-02-09 14:54:31,365 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/d1e0c6c3f72940668e505a5227f8b1bb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/d1e0c6c3f72940668e505a5227f8b1bb 2023-02-09 14:54:31,366 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/f267f4025fb34ba3ab5ddd61ba69810b to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf1/f267f4025fb34ba3ab5ddd61ba69810b 2023-02-09 14:54:31,369 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/3259f9dfc059461f86903f033c01ca31 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/3259f9dfc059461f86903f033c01ca31 2023-02-09 14:54:31,370 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/39a737174efa406f8685bd11869876a4 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/39a737174efa406f8685bd11869876a4 2023-02-09 14:54:31,371 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/6adf8d085dcc438aba69554d90273646 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/6adf8d085dcc438aba69554d90273646 2023-02-09 14:54:31,372 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/6c84bb9704ec4bfcbadfb6d74ba97d77 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/6c84bb9704ec4bfcbadfb6d74ba97d77 2023-02-09 14:54:31,373 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/7058f4e7029c4e35912b425b094d4fb4 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/7058f4e7029c4e35912b425b094d4fb4 2023-02-09 14:54:31,374 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/803c92c516a94497bd8e320dd6fd8469 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/803c92c516a94497bd8e320dd6fd8469 2023-02-09 14:54:31,375 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/92a9fcb53ed146f58cd20ee6828e019a to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/92a9fcb53ed146f58cd20ee6828e019a 2023-02-09 14:54:31,376 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/9974ec9e406648e58660186bb1d7f420 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/9974ec9e406648e58660186bb1d7f420 2023-02-09 14:54:31,378 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/aaf025f0f70e4a3e839770e623098e97 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/aaf025f0f70e4a3e839770e623098e97 2023-02-09 14:54:31,379 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/ca9fd58be3704739a1150ab8f4f79ea4 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/cf2/ca9fd58be3704739a1150ab8f4f79ea4 2023-02-09 14:54:31,381 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/recovered.edits/106.seqid to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78/recovered.edits/106.seqid 2023-02-09 14:54:31,382 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/94ffc82cbf109e7f71870e19837a0e78 2023-02-09 14:54:31,382 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-09 14:54:31,384 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:31,386 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-02-09 14:54:31,388 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-02-09 14:54:31,389 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:31,389 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-02-09 14:54:31,389 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954471389"}]},"ts":"9223372036854775807"} 2023-02-09 14:54:31,389 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954471389"}]},"ts":"9223372036854775807"} 2023-02-09 14:54:31,389 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954471389"}]},"ts":"9223372036854775807"} 2023-02-09 14:54:31,390 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 14:54:31,390 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 94ffc82cbf109e7f71870e19837a0e78, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675954467691.94ffc82cbf109e7f71870e19837a0e78.', STARTKEY => '', ENDKEY => ''}, {ENCODED => c720e43e6c74641ba50283474b45a523, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1675954469873.c720e43e6c74641ba50283474b45a523.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => 8c23d3f07541daba404c2b3d3332d0cb, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1675954469873.8c23d3f07541daba404c2b3d3332d0cb.', STARTKEY => '60', ENDKEY => ''}] 2023-02-09 14:54:31,390 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-02-09 14:54:31,390 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675954471390"}]},"ts":"9223372036854775807"} 2023-02-09 14:54:31,391 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-02-09 14:54:31,394 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:54:31,395 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 51 msec 2023-02-09 14:54:31,446 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:31,446 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:31,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-09 14:54:31,447 INFO [Listener at localhost/40823] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-02-09 14:54:31,462 INFO [Listener at localhost/40823] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=407 (was 402) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:35198 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:34646 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:35208 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:42500 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:42606 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:37024 [Waiting for operation #47] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:42564 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:39044 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:35180 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:42570 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:35182 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:42532 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:39078 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:35224 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x7e2f5f25-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:37030 [Waiting for operation #41] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_153774153_15 at /127.0.0.1:42518 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=857 (was 855) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=250 (was 237) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3147 (was 3213) 2023-02-09 14:54:31,476 INFO [Listener at localhost/40823] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=407, OpenFileDescriptor=857, MaxFileDescriptor=60000, SystemLoadAverage=250, ProcessCount=168, AvailableMemoryMB=3146 2023-02-09 14:54:31,476 WARN [Listener at localhost/40823] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 14:54:31,477 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:31,477 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:31,479 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 14:54:31,479 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 14:54:31,480 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:54:31,482 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 14:54:31,482 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 14:54:31,484 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/080674692710b4dfd3481fd1d48b1c60 2023-02-09 14:54:31,484 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/080674692710b4dfd3481fd1d48b1c60 empty. 2023-02-09 14:54:31,484 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/080674692710b4dfd3481fd1d48b1c60 2023-02-09 14:54:31,484 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-09 14:54:31,493 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-09 14:54:31,494 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7671): creating {ENCODED => 080674692710b4dfd3481fd1d48b1c60, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp 2023-02-09 14:54:31,501 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:31,501 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1603): Closing 080674692710b4dfd3481fd1d48b1c60, disabling compactions & flushes 2023-02-09 14:54:31,501 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60. 2023-02-09 14:54:31,501 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60. 2023-02-09 14:54:31,501 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60. after waiting 0 ms 2023-02-09 14:54:31,501 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60. 2023-02-09 14:54:31,501 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60. 2023-02-09 14:54:31,501 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1557): Region close journal for 080674692710b4dfd3481fd1d48b1c60: Waiting for close lock at 1675954471501Disabling compacts and flushes for region at 1675954471501Disabling writes for close at 1675954471501Writing region close event to WAL at 1675954471501Closed at 1675954471501 2023-02-09 14:54:31,503 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 14:54:31,504 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675954471504"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954471504"}]},"ts":"1675954471504"} 2023-02-09 14:54:31,505 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 14:54:31,506 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 14:54:31,506 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954471506"}]},"ts":"1675954471506"} 2023-02-09 14:54:31,507 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-02-09 14:54:31,510 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 14:54:31,511 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 14:54:31,511 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 14:54:31,511 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 14:54:31,511 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 14:54:31,511 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=080674692710b4dfd3481fd1d48b1c60, ASSIGN}] 2023-02-09 14:54:31,513 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=080674692710b4dfd3481fd1d48b1c60, ASSIGN 2023-02-09 14:54:31,514 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=080674692710b4dfd3481fd1d48b1c60, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33423,1675954443133; forceNewPlan=false, retain=false 2023-02-09 14:54:31,664 INFO [jenkins-hbase4:43829] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 14:54:31,667 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=080674692710b4dfd3481fd1d48b1c60, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:31,667 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675954471667"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954471667"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954471667"}]},"ts":"1675954471667"} 2023-02-09 14:54:31,668 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 080674692710b4dfd3481fd1d48b1c60, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:54:31,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60. 2023-02-09 14:54:31,822 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 080674692710b4dfd3481fd1d48b1c60, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:54:31,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:54:31,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 080674692710b4dfd3481fd1d48b1c60 2023-02-09 14:54:31,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:31,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 080674692710b4dfd3481fd1d48b1c60 2023-02-09 14:54:31,823 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 080674692710b4dfd3481fd1d48b1c60 2023-02-09 14:54:31,824 INFO [StoreOpener-080674692710b4dfd3481fd1d48b1c60-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 080674692710b4dfd3481fd1d48b1c60 2023-02-09 14:54:31,825 DEBUG [StoreOpener-080674692710b4dfd3481fd1d48b1c60-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackForSplitTableRegionWithReplica/080674692710b4dfd3481fd1d48b1c60/cf1 2023-02-09 14:54:31,825 DEBUG [StoreOpener-080674692710b4dfd3481fd1d48b1c60-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackForSplitTableRegionWithReplica/080674692710b4dfd3481fd1d48b1c60/cf1 2023-02-09 14:54:31,826 INFO [StoreOpener-080674692710b4dfd3481fd1d48b1c60-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 080674692710b4dfd3481fd1d48b1c60 columnFamilyName cf1 2023-02-09 14:54:31,826 INFO [StoreOpener-080674692710b4dfd3481fd1d48b1c60-1] regionserver.HStore(310): Store=080674692710b4dfd3481fd1d48b1c60/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:31,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackForSplitTableRegionWithReplica/080674692710b4dfd3481fd1d48b1c60 2023-02-09 14:54:31,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackForSplitTableRegionWithReplica/080674692710b4dfd3481fd1d48b1c60 2023-02-09 14:54:31,829 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 080674692710b4dfd3481fd1d48b1c60 2023-02-09 14:54:31,831 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackForSplitTableRegionWithReplica/080674692710b4dfd3481fd1d48b1c60/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:54:31,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 080674692710b4dfd3481fd1d48b1c60; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67647569, jitterRate=0.008027330040931702}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 14:54:31,831 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 080674692710b4dfd3481fd1d48b1c60: Running coprocessor pre-open hook at 1675954471823Writing region info on filesystem at 1675954471823Initializing all the Stores at 1675954471824 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954471824Cleaning up temporary data from old regions at 1675954471827 (+3 ms)Cleaning up detritus from prior splits at 1675954471828 (+1 ms)Running coprocessor post-open hooks at 1675954471831 (+3 ms)Region opened successfully at 1675954471831 2023-02-09 14:54:31,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60., pid=87, masterSystemTime=1675954471820 2023-02-09 14:54:31,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60. 2023-02-09 14:54:31,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60. 2023-02-09 14:54:31,833 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=080674692710b4dfd3481fd1d48b1c60, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:31,833 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675954471833"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954471833"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954471833"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954471833"}]},"ts":"1675954471833"} 2023-02-09 14:54:31,836 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-02-09 14:54:31,836 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 080674692710b4dfd3481fd1d48b1c60, server=jenkins-hbase4.apache.org,33423,1675954443133 in 166 msec 2023-02-09 14:54:31,837 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-02-09 14:54:31,837 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=080674692710b4dfd3481fd1d48b1c60, ASSIGN in 325 msec 2023-02-09 14:54:31,838 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 14:54:31,838 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954471838"}]},"ts":"1675954471838"} 2023-02-09 14:54:31,839 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-02-09 14:54:31,848 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 14:54:31,849 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 368 msec 2023-02-09 14:54:31,981 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:31,981 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:31,983 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:54:31,983 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:54:31,990 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-09 14:54:31,997 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:54:32,008 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-09 14:54:32,009 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-09 14:54:32,009 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-09 14:54:32,012 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-02-09 14:54:32,023 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=080674692710b4dfd3481fd1d48b1c60, REOPEN/MOVE}] 2023-02-09 14:54:32,024 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=080674692710b4dfd3481fd1d48b1c60, REOPEN/MOVE 2023-02-09 14:54:32,025 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=080674692710b4dfd3481fd1d48b1c60, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:32,025 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675954472025"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954472025"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954472025"}]},"ts":"1675954472025"} 2023-02-09 14:54:32,026 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 080674692710b4dfd3481fd1d48b1c60, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:54:32,178 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 080674692710b4dfd3481fd1d48b1c60 2023-02-09 14:54:32,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 080674692710b4dfd3481fd1d48b1c60, disabling compactions & flushes 2023-02-09 14:54:32,180 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60. 2023-02-09 14:54:32,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60. 2023-02-09 14:54:32,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60. after waiting 0 ms 2023-02-09 14:54:32,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60. 2023-02-09 14:54:32,183 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackForSplitTableRegionWithReplica/080674692710b4dfd3481fd1d48b1c60/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-09 14:54:32,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:54:32,184 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60. 2023-02-09 14:54:32,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 080674692710b4dfd3481fd1d48b1c60: Waiting for close lock at 1675954472179Running coprocessor pre-close hooks at 1675954472179Disabling compacts and flushes for region at 1675954472179Disabling writes for close at 1675954472180 (+1 ms)Writing region close event to WAL at 1675954472180Running coprocessor post-close hooks at 1675954472184 (+4 ms)Closed at 1675954472184 2023-02-09 14:54:32,184 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 080674692710b4dfd3481fd1d48b1c60 to self. 2023-02-09 14:54:32,185 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 080674692710b4dfd3481fd1d48b1c60 2023-02-09 14:54:32,186 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=080674692710b4dfd3481fd1d48b1c60, regionState=CLOSED 2023-02-09 14:54:32,186 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675954472186"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954472186"}]},"ts":"1675954472186"} 2023-02-09 14:54:32,188 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-02-09 14:54:32,189 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 080674692710b4dfd3481fd1d48b1c60, server=jenkins-hbase4.apache.org,33423,1675954443133 in 161 msec 2023-02-09 14:54:32,189 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=080674692710b4dfd3481fd1d48b1c60, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,33423,1675954443133; forceNewPlan=false, retain=true 2023-02-09 14:54:32,339 INFO [jenkins-hbase4:43829] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 14:54:32,340 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=080674692710b4dfd3481fd1d48b1c60, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:32,340 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675954472340"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954472340"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954472340"}]},"ts":"1675954472340"} 2023-02-09 14:54:32,341 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 080674692710b4dfd3481fd1d48b1c60, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:54:32,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60. 2023-02-09 14:54:32,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 080674692710b4dfd3481fd1d48b1c60, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:54:32,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:54:32,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 080674692710b4dfd3481fd1d48b1c60 2023-02-09 14:54:32,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:32,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 080674692710b4dfd3481fd1d48b1c60 2023-02-09 14:54:32,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 080674692710b4dfd3481fd1d48b1c60 2023-02-09 14:54:32,498 INFO [StoreOpener-080674692710b4dfd3481fd1d48b1c60-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 080674692710b4dfd3481fd1d48b1c60 2023-02-09 14:54:32,499 DEBUG [StoreOpener-080674692710b4dfd3481fd1d48b1c60-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackForSplitTableRegionWithReplica/080674692710b4dfd3481fd1d48b1c60/cf1 2023-02-09 14:54:32,499 DEBUG [StoreOpener-080674692710b4dfd3481fd1d48b1c60-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackForSplitTableRegionWithReplica/080674692710b4dfd3481fd1d48b1c60/cf1 2023-02-09 14:54:32,499 INFO [StoreOpener-080674692710b4dfd3481fd1d48b1c60-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 080674692710b4dfd3481fd1d48b1c60 columnFamilyName cf1 2023-02-09 14:54:32,500 INFO [StoreOpener-080674692710b4dfd3481fd1d48b1c60-1] regionserver.HStore(310): Store=080674692710b4dfd3481fd1d48b1c60/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:54:32,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackForSplitTableRegionWithReplica/080674692710b4dfd3481fd1d48b1c60 2023-02-09 14:54:32,501 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackForSplitTableRegionWithReplica/080674692710b4dfd3481fd1d48b1c60 2023-02-09 14:54:32,503 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 080674692710b4dfd3481fd1d48b1c60 2023-02-09 14:54:32,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 080674692710b4dfd3481fd1d48b1c60; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70011354, jitterRate=0.043250471353530884}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 14:54:32,504 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 080674692710b4dfd3481fd1d48b1c60: Running coprocessor pre-open hook at 1675954472496Writing region info on filesystem at 1675954472496Initializing all the Stores at 1675954472497 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954472497Cleaning up temporary data from old regions at 1675954472502 (+5 ms)Cleaning up detritus from prior splits at 1675954472502Running coprocessor post-open hooks at 1675954472504 (+2 ms)Region opened successfully at 1675954472504 2023-02-09 14:54:32,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60., pid=92, masterSystemTime=1675954472493 2023-02-09 14:54:32,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60. 2023-02-09 14:54:32,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60. 2023-02-09 14:54:32,507 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=080674692710b4dfd3481fd1d48b1c60, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:54:32,507 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675954472507"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954472507"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954472507"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954472507"}]},"ts":"1675954472507"} 2023-02-09 14:54:32,510 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-09 14:54:32,510 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 080674692710b4dfd3481fd1d48b1c60, server=jenkins-hbase4.apache.org,33423,1675954443133 in 167 msec 2023-02-09 14:54:32,511 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-02-09 14:54:32,511 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=080674692710b4dfd3481fd1d48b1c60, REOPEN/MOVE in 487 msec 2023-02-09 14:54:32,514 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-09 14:54:32,514 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 502 msec 2023-02-09 14:54:32,516 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=813d8870e2a4f2929f4991fa0f505d12, ASSIGN}] 2023-02-09 14:54:32,517 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=813d8870e2a4f2929f4991fa0f505d12, ASSIGN 2023-02-09 14:54:32,518 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=813d8870e2a4f2929f4991fa0f505d12, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-09 14:54:32,668 DEBUG [jenkins-hbase4:43829] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 14:54:32,669 DEBUG [jenkins-hbase4:43829] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 14:54:32,669 DEBUG [jenkins-hbase4:43829] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 14:54:32,669 DEBUG [jenkins-hbase4:43829] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 14:54:32,670 DEBUG [jenkins-hbase4:43829] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-02-09 14:54:32,671 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=813d8870e2a4f2929f4991fa0f505d12, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:54:32,671 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675954472671"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1675954472671"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1675954472671"}]},"ts":"1675954472671"} 2023-02-09 14:54:32,675 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure 813d8870e2a4f2929f4991fa0f505d12, server=jenkins-hbase4.apache.org,41077,1675954443212}] 2023-02-09 14:54:32,829 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1675954471479_0001.813d8870e2a4f2929f4991fa0f505d12. 2023-02-09 14:54:32,829 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 813d8870e2a4f2929f4991fa0f505d12, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675954471479_0001.813d8870e2a4f2929f4991fa0f505d12.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-02-09 14:54:32,829 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:54:32,830 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 813d8870e2a4f2929f4991fa0f505d12 2023-02-09 14:54:32,830 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1675954471479_0001.813d8870e2a4f2929f4991fa0f505d12.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:54:32,830 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 813d8870e2a4f2929f4991fa0f505d12 2023-02-09 14:54:32,830 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 813d8870e2a4f2929f4991fa0f505d12 2023-02-09 14:54:32,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:32,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:33,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:33,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:33,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:33,331 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:33,431 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:33,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:33,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:33,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:33,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:33,932 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:34,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:34,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:34,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:34,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:34,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:34,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:34,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:34,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:34,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:34,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:35,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:35,045 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-02-09 14:54:35,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:35,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:35,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:35,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:35,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:35,573 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 14:54:35,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:35,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:35,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:35,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:36,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:36,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:36,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:36,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:36,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:36,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:36,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:36,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:36,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:36,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:37,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:37,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:37,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:37,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:37,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:37,539 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:37,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:37,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:37,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:37,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:38,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:38,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:38,240 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:38,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:38,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:38,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:38,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:38,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:38,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:38,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:39,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:39,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:39,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:39,342 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:39,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:39,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:39,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:39,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:39,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:39,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:40,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:40,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:40,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:40,344 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:40,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:40,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:40,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:40,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:40,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:40,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:41,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:41,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:41,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:41,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:41,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:41,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:41,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:41,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:41,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:41,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:42,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:42,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:42,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:42,348 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:42,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:42,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:42,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:42,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:42,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:42,949 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:43,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:43,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:43,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:43,350 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:43,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:43,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:43,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:43,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:43,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:43,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:44,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:44,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:44,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:44,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:44,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:44,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:44,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:44,753 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:44,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:44,953 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:45,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:45,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:45,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:45,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:45,454 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:45,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:45,655 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:45,755 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:45,855 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:45,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:46,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:46,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:46,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:46,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:46,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:46,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:46,657 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:46,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:46,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:46,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:47,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:47,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:47,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:47,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:47,458 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:47,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:47,659 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:47,759 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:47,859 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:47,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:48,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:48,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:48,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:48,360 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:48,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:48,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:48,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:48,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:48,861 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:48,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:49,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:49,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:49,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:49,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:49,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:49,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:49,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:49,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:49,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:49,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:50,063 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:50,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:50,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:50,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:50,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:50,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:50,665 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:50,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:50,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:50,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:51,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:51,165 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:51,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:51,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:51,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:51,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:51,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:51,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:51,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:51,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:52,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:52,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:52,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:52,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:52,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:52,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:52,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:52,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:52,869 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:52,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:53,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:53,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:53,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:53,370 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:53,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:53,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:53,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:53,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:53,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:53,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:54,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:54,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:54,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:54,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:54,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:54,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:54,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:54,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:54,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:54,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:55,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:55,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:55,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:55,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:55,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:55,574 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:55,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:55,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:55,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:55,975 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:56,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:56,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:56,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:56,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:56,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:56,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:56,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:56,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:56,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:56,977 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:57,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:57,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:57,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:57,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:57,478 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:57,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:57,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:57,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:57,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:57,979 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:58,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:58,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:58,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:58,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:58,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:58,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:58,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:58,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:58,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:58,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:59,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:59,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:59,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:59,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:59,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:59,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:59,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:59,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:59,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:54:59,983 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:00,083 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:00,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:00,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:00,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:00,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:00,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:00,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:00,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:00,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:00,985 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:01,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:01,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:01,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:01,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:01,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:01,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:01,686 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:01,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:01,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:01,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:02,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:02,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:02,288 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:02,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:02,488 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:02,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:02,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:02,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:02,889 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:02,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:03,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:03,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:03,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:03,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:03,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:03,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:03,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:03,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:03,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:03,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:04,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:04,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:04,292 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:04,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:04,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:04,592 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:04,632 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-09 14:55:04,636 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-09 14:55:04,689 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-09 14:55:04,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:04,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:04,893 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:04,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:05,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:05,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:05,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:05,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:05,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:05,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:05,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:05,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:05,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:05,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:06,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:06,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:06,295 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:06,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:06,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:06,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:06,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:06,796 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:06,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:06,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:07,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:07,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:07,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:07,397 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:07,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:07,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:07,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:07,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:07,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:07,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:08,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:08,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:08,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:08,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:08,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:08,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:08,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:08,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:08,900 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:09,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:09,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:09,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:09,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:09,401 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:09,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:09,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:09,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:09,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:09,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:10,002 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:10,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:10,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:10,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:10,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:10,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:10,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:10,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:10,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:10,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:11,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:11,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:11,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:11,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:11,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:11,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:11,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:11,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:11,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:11,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:12,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:12,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:12,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:12,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:12,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:12,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:12,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:12,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:12,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:12,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:13,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:13,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:13,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:13,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:13,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:13,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:13,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:13,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:13,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:13,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:14,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:14,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:14,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:14,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:14,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:14,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:14,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:14,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:14,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:14,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:15,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:15,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:15,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:15,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:15,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:15,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:15,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:15,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:15,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:15,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:16,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:16,114 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:16,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:16,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:16,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:16,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:16,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:16,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:16,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:16,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:17,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:17,116 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:17,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:17,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:17,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:17,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:17,617 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:17,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:17,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:17,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:18,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:18,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:18,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:18,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:18,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:18,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:18,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:18,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:18,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:18,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:19,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:19,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:19,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:19,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:19,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:19,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:19,621 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:19,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:19,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:19,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:20,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:20,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:20,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:20,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:20,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:20,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:20,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:20,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:20,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:20,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:21,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:21,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:21,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:21,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:21,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:21,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:21,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:21,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:21,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:21,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:22,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:22,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:22,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:22,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:22,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:22,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:22,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:22,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:22,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:22,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:23,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:23,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:23,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:23,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:23,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:23,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:23,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:23,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:23,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:23,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:24,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:24,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:24,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:24,331 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:24,431 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:24,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:24,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:24,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:24,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:24,932 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:25,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:25,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:25,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:25,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:25,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:25,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:25,634 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:25,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:25,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:25,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:26,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:26,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:26,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:26,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:26,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:26,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:26,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:26,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:26,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:26,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:27,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:27,137 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:27,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:27,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:27,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:27,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:27,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:27,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:27,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:27,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:28,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:28,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:28,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:28,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:28,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:28,539 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:28,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:28,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:28,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:28,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:29,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:29,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:29,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:29,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:29,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:29,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:29,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:29,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:29,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:29,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:30,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:30,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:30,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:30,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:30,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:30,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:30,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:30,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:30,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:30,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:31,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:31,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:31,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:31,345 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:31,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:31,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:31,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:31,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:31,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:31,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:32,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:32,102 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:32,102 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:32,105 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:32,105 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:32,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 14:55:32,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:32,206 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:32,206 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:32,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 14:55:32,247 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:32,347 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:32,407 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:32,408 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:32,409 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 14:55:32,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:32,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:32,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:32,709 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:32,709 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:32,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 14:55:32,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:32,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:32,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:33,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:33,149 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:33,210 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:33,210 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:33,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-09 14:55:33,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:33,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:33,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:33,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:33,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:33,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:33,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:33,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:34,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:34,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:34,213 DEBUG [Listener at localhost/40823] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:34,214 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=080674692710b4dfd3481fd1d48b1c60, daughterA=45103f245599b3c2b8e9be2a40c7083b, daughterB=47b175695c3d18e949263cbc873d2e48 2023-02-09 14:55:34,215 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=080674692710b4dfd3481fd1d48b1c60, daughterA=45103f245599b3c2b8e9be2a40c7083b, daughterB=47b175695c3d18e949263cbc873d2e48 2023-02-09 14:55:34,215 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=080674692710b4dfd3481fd1d48b1c60, daughterA=45103f245599b3c2b8e9be2a40c7083b, daughterB=47b175695c3d18e949263cbc873d2e48 2023-02-09 14:55:34,215 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=080674692710b4dfd3481fd1d48b1c60, daughterA=45103f245599b3c2b8e9be2a40c7083b, daughterB=47b175695c3d18e949263cbc873d2e48 2023-02-09 14:55:34,217 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 080674692710b4dfd3481fd1d48b1c60, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=080674692710b4dfd3481fd1d48b1c60, daughterA=45103f245599b3c2b8e9be2a40c7083b, daughterB=47b175695c3d18e949263cbc873d2e48 org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41077,1675954443212, table=testRollbackForSplitTableRegionWithReplica, region=813d8870e2a4f2929f4991fa0f505d12 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:55:34,219 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41077,1675954443212, table=testRollbackForSplitTableRegionWithReplica, region=813d8870e2a4f2929f4991fa0f505d12 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=080674692710b4dfd3481fd1d48b1c60, daughterA=45103f245599b3c2b8e9be2a40c7083b, daughterB=47b175695c3d18e949263cbc873d2e48 exec-time=5 msec 2023-02-09 14:55:34,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:34,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:34,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-09 14:55:34,552 INFO [StoreOpener-813d8870e2a4f2929f4991fa0f505d12-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 813d8870e2a4f2929f4991fa0f505d12 2023-02-09 14:55:34,555 DEBUG [StoreOpener-813d8870e2a4f2929f4991fa0f505d12-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackForSplitTableRegionWithReplica/080674692710b4dfd3481fd1d48b1c60/cf1 2023-02-09 14:55:34,555 DEBUG [StoreOpener-813d8870e2a4f2929f4991fa0f505d12-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackForSplitTableRegionWithReplica/080674692710b4dfd3481fd1d48b1c60/cf1 2023-02-09 14:55:34,556 INFO [StoreOpener-813d8870e2a4f2929f4991fa0f505d12-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 813d8870e2a4f2929f4991fa0f505d12 columnFamilyName cf1 2023-02-09 14:55:34,556 INFO [StoreOpener-813d8870e2a4f2929f4991fa0f505d12-1] regionserver.HStore(310): Store=813d8870e2a4f2929f4991fa0f505d12/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:34,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 813d8870e2a4f2929f4991fa0f505d12; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64907615, jitterRate=-0.03280116617679596}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-09 14:55:34,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 813d8870e2a4f2929f4991fa0f505d12: Running coprocessor pre-open hook at 1675954472830Initializing all the Stores at 1675954534551 (+61721 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954534551Running coprocessor post-open hooks at 1675954534558 (+7 ms)Region opened successfully at 1675954534558 2023-02-09 14:55:34,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1675954471479_0001.813d8870e2a4f2929f4991fa0f505d12., pid=94, masterSystemTime=1675954472826 2023-02-09 14:55:34,559 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1675954471479_0001.813d8870e2a4f2929f4991fa0f505d12. 2023-02-09 14:55:34,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1675954471479_0001.813d8870e2a4f2929f4991fa0f505d12. 2023-02-09 14:55:34,560 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=813d8870e2a4f2929f4991fa0f505d12, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:34,560 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675954534560"},{"qualifier":"server_0001","vlen":31,"tag":[],"timestamp":"1675954534560"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1675954534560"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1675954534560"}]},"ts":"1675954534560"} 2023-02-09 14:55:34,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-02-09 14:55:34,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure 813d8870e2a4f2929f4991fa0f505d12, server=jenkins-hbase4.apache.org,41077,1675954443212 in 1 mins, 1.888 sec 2023-02-09 14:55:34,564 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-02-09 14:55:34,564 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=813d8870e2a4f2929f4991fa0f505d12, ASSIGN in 1 mins, 2.047 sec 2023-02-09 14:55:34,566 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.573 sec 2023-02-09 14:55:34,566 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1675954471987","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62579,"client":"172.31.14.131:49848","queuetimems":0,"class":"HMaster"} 2023-02-09 14:55:34,567 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 601, skipping over this response of 3 bytes 2023-02-09 14:55:36,465 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41077,1675954443212, table=testRollbackForSplitTableRegionWithReplica, region=813d8870e2a4f2929f4991fa0f505d12 is currently in transition, give up 2023-02-09 14:55:36,466 WARN [Listener at localhost/40823] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 14:55:36,466 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:36,466 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:36,467 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:36,468 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:36,468 INFO [Listener at localhost/40823] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:36,468 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:36,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:36,470 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:36,471 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:36,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-09 14:55:36,471 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954536471"}]},"ts":"1675954536471"} 2023-02-09 14:55:36,473 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-02-09 14:55:36,474 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-02-09 14:55:36,475 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=080674692710b4dfd3481fd1d48b1c60, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=813d8870e2a4f2929f4991fa0f505d12, UNASSIGN}] 2023-02-09 14:55:36,475 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=813d8870e2a4f2929f4991fa0f505d12, UNASSIGN 2023-02-09 14:55:36,476 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=813d8870e2a4f2929f4991fa0f505d12, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:36,476 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675954536476"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1675954536476"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1675954536476"}]},"ts":"1675954536476"} 2023-02-09 14:55:36,477 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure 813d8870e2a4f2929f4991fa0f505d12, server=jenkins-hbase4.apache.org,41077,1675954443212}] 2023-02-09 14:55:36,479 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=080674692710b4dfd3481fd1d48b1c60, UNASSIGN 2023-02-09 14:55:36,479 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=080674692710b4dfd3481fd1d48b1c60, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:36,480 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675954536479"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954536479"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954536479"}]},"ts":"1675954536479"} 2023-02-09 14:55:36,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 080674692710b4dfd3481fd1d48b1c60, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:55:36,571 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:36,572 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:36,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-09 14:55:36,629 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 813d8870e2a4f2929f4991fa0f505d12 2023-02-09 14:55:36,630 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 813d8870e2a4f2929f4991fa0f505d12, disabling compactions & flushes 2023-02-09 14:55:36,630 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675954471479_0001.813d8870e2a4f2929f4991fa0f505d12. 2023-02-09 14:55:36,630 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675954471479_0001.813d8870e2a4f2929f4991fa0f505d12. 2023-02-09 14:55:36,630 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675954471479_0001.813d8870e2a4f2929f4991fa0f505d12. after waiting 0 ms 2023-02-09 14:55:36,630 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675954471479_0001.813d8870e2a4f2929f4991fa0f505d12. 2023-02-09 14:55:36,630 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:55:36,631 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675954471479_0001.813d8870e2a4f2929f4991fa0f505d12. 2023-02-09 14:55:36,631 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 813d8870e2a4f2929f4991fa0f505d12: Waiting for close lock at 1675954536629Running coprocessor pre-close hooks at 1675954536629Disabling compacts and flushes for region at 1675954536629Disabling writes for close at 1675954536630 (+1 ms)Writing region close event to WAL at 1675954536630Running coprocessor post-close hooks at 1675954536630Closed at 1675954536631 (+1 ms) 2023-02-09 14:55:36,632 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 080674692710b4dfd3481fd1d48b1c60 2023-02-09 14:55:36,633 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 080674692710b4dfd3481fd1d48b1c60, disabling compactions & flushes 2023-02-09 14:55:36,633 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60. 2023-02-09 14:55:36,633 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60. 2023-02-09 14:55:36,633 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60. after waiting 0 ms 2023-02-09 14:55:36,633 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60. 2023-02-09 14:55:36,633 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 813d8870e2a4f2929f4991fa0f505d12 2023-02-09 14:55:36,634 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=813d8870e2a4f2929f4991fa0f505d12, regionState=CLOSED 2023-02-09 14:55:36,634 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675954536634"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1675954536634"}]},"ts":"1675954536634"} 2023-02-09 14:55:36,637 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRollbackForSplitTableRegionWithReplica/080674692710b4dfd3481fd1d48b1c60/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-09 14:55:36,637 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-02-09 14:55:36,638 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:55:36,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure 813d8870e2a4f2929f4991fa0f505d12, server=jenkins-hbase4.apache.org,41077,1675954443212 in 159 msec 2023-02-09 14:55:36,638 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60. 2023-02-09 14:55:36,638 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 080674692710b4dfd3481fd1d48b1c60: Waiting for close lock at 1675954536632Running coprocessor pre-close hooks at 1675954536632Disabling compacts and flushes for region at 1675954536632Disabling writes for close at 1675954536633 (+1 ms)Writing region close event to WAL at 1675954536633Running coprocessor post-close hooks at 1675954536638 (+5 ms)Closed at 1675954536638 2023-02-09 14:55:36,639 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 080674692710b4dfd3481fd1d48b1c60 2023-02-09 14:55:36,639 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=813d8870e2a4f2929f4991fa0f505d12, UNASSIGN in 163 msec 2023-02-09 14:55:36,640 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=080674692710b4dfd3481fd1d48b1c60, regionState=CLOSED 2023-02-09 14:55:36,640 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1675954536640"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954536640"}]},"ts":"1675954536640"} 2023-02-09 14:55:36,642 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-02-09 14:55:36,642 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 080674692710b4dfd3481fd1d48b1c60, server=jenkins-hbase4.apache.org,33423,1675954443133 in 161 msec 2023-02-09 14:55:36,644 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-02-09 14:55:36,644 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=080674692710b4dfd3481fd1d48b1c60, UNASSIGN in 167 msec 2023-02-09 14:55:36,644 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954536644"}]},"ts":"1675954536644"} 2023-02-09 14:55:36,645 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-02-09 14:55:36,646 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-02-09 14:55:36,647 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 179 msec 2023-02-09 14:55:36,772 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:36,773 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:36,773 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-09 14:55:36,773 INFO [Listener at localhost/40823] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-02-09 14:55:36,773 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:36,773 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:36,774 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:36,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:36,776 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:36,776 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:36,776 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:36,777 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:36,777 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-09 14:55:36,781 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/080674692710b4dfd3481fd1d48b1c60 2023-02-09 14:55:36,782 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/080674692710b4dfd3481fd1d48b1c60/cf1, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/080674692710b4dfd3481fd1d48b1c60/recovered.edits] 2023-02-09 14:55:36,786 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/080674692710b4dfd3481fd1d48b1c60/recovered.edits/7.seqid to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRollbackForSplitTableRegionWithReplica/080674692710b4dfd3481fd1d48b1c60/recovered.edits/7.seqid 2023-02-09 14:55:36,787 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/080674692710b4dfd3481fd1d48b1c60 2023-02-09 14:55:36,787 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-09 14:55:36,789 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:36,790 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-02-09 14:55:36,792 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-02-09 14:55:36,793 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:36,793 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-02-09 14:55:36,793 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954536793"}]},"ts":"9223372036854775807"} 2023-02-09 14:55:36,793 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1675954471479_0001.813d8870e2a4f2929f4991fa0f505d12.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954536793"}]},"ts":"9223372036854775807"} 2023-02-09 14:55:36,794 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-02-09 14:55:36,794 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 080674692710b4dfd3481fd1d48b1c60, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675954471479.080674692710b4dfd3481fd1d48b1c60.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 813d8870e2a4f2929f4991fa0f505d12, NAME => 'testRollbackForSplitTableRegionWithReplica,,1675954471479_0001.813d8870e2a4f2929f4991fa0f505d12.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-02-09 14:55:36,794 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-02-09 14:55:36,794 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675954536794"}]},"ts":"9223372036854775807"} 2023-02-09 14:55:36,796 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-02-09 14:55:36,799 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:36,799 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 25 msec 2023-02-09 14:55:36,878 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:36,878 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:36,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-09 14:55:36,879 INFO [Listener at localhost/40823] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-02-09 14:55:36,891 INFO [Listener at localhost/40823] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=340 (was 407), OpenFileDescriptor=652 (was 857), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=91 (was 250), ProcessCount=168 (was 168), AvailableMemoryMB=2981 (was 3146) 2023-02-09 14:55:36,903 INFO [Listener at localhost/40823] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=341, OpenFileDescriptor=652, MaxFileDescriptor=60000, SystemLoadAverage=91, ProcessCount=168, AvailableMemoryMB=2981 2023-02-09 14:55:36,903 WARN [Listener at localhost/40823] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 14:55:36,903 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:36,903 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:36,905 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 14:55:36,905 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 14:55:36,906 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:36,908 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 14:55:36,908 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 14:55:36,910 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99 2023-02-09 14:55:36,910 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99 empty. 2023-02-09 14:55:36,911 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99 2023-02-09 14:55:36,911 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-09 14:55:36,921 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-02-09 14:55:36,922 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7671): creating {ENCODED => 7f4736b52b7063d2389b5410b3a30a99, NAME => 'testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp 2023-02-09 14:55:36,932 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:36,932 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1603): Closing 7f4736b52b7063d2389b5410b3a30a99, disabling compactions & flushes 2023-02-09 14:55:36,932 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. 2023-02-09 14:55:36,932 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. 2023-02-09 14:55:36,932 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. after waiting 0 ms 2023-02-09 14:55:36,933 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. 2023-02-09 14:55:36,933 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. 2023-02-09 14:55:36,933 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1557): Region close journal for 7f4736b52b7063d2389b5410b3a30a99: Waiting for close lock at 1675954536932Disabling compacts and flushes for region at 1675954536932Disabling writes for close at 1675954536933 (+1 ms)Writing region close event to WAL at 1675954536933Closed at 1675954536933 2023-02-09 14:55:36,934 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 14:55:36,935 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675954536935"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954536935"}]},"ts":"1675954536935"} 2023-02-09 14:55:36,936 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 14:55:36,937 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 14:55:36,937 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954536937"}]},"ts":"1675954536937"} 2023-02-09 14:55:36,938 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-02-09 14:55:36,940 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 14:55:36,941 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 14:55:36,941 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 14:55:36,941 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 14:55:36,941 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 14:55:36,942 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=7f4736b52b7063d2389b5410b3a30a99, ASSIGN}] 2023-02-09 14:55:36,943 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=7f4736b52b7063d2389b5410b3a30a99, ASSIGN 2023-02-09 14:55:36,943 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=7f4736b52b7063d2389b5410b3a30a99, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33423,1675954443133; forceNewPlan=false, retain=false 2023-02-09 14:55:37,094 INFO [jenkins-hbase4:43829] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 14:55:37,095 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=7f4736b52b7063d2389b5410b3a30a99, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,095 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675954537095"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954537095"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954537095"}]},"ts":"1675954537095"} 2023-02-09 14:55:37,097 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure 7f4736b52b7063d2389b5410b3a30a99, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:55:37,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. 2023-02-09 14:55:37,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7f4736b52b7063d2389b5410b3a30a99, NAME => 'testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:55:37,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:55:37,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey 7f4736b52b7063d2389b5410b3a30a99 2023-02-09 14:55:37,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:37,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 7f4736b52b7063d2389b5410b3a30a99 2023-02-09 14:55:37,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 7f4736b52b7063d2389b5410b3a30a99 2023-02-09 14:55:37,253 INFO [StoreOpener-7f4736b52b7063d2389b5410b3a30a99-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7f4736b52b7063d2389b5410b3a30a99 2023-02-09 14:55:37,254 DEBUG [StoreOpener-7f4736b52b7063d2389b5410b3a30a99-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1 2023-02-09 14:55:37,254 DEBUG [StoreOpener-7f4736b52b7063d2389b5410b3a30a99-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1 2023-02-09 14:55:37,255 INFO [StoreOpener-7f4736b52b7063d2389b5410b3a30a99-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7f4736b52b7063d2389b5410b3a30a99 columnFamilyName cf1 2023-02-09 14:55:37,255 INFO [StoreOpener-7f4736b52b7063d2389b5410b3a30a99-1] regionserver.HStore(310): Store=7f4736b52b7063d2389b5410b3a30a99/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:37,255 INFO [StoreOpener-7f4736b52b7063d2389b5410b3a30a99-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7f4736b52b7063d2389b5410b3a30a99 2023-02-09 14:55:37,256 DEBUG [StoreOpener-7f4736b52b7063d2389b5410b3a30a99-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2 2023-02-09 14:55:37,257 DEBUG [StoreOpener-7f4736b52b7063d2389b5410b3a30a99-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2 2023-02-09 14:55:37,257 INFO [StoreOpener-7f4736b52b7063d2389b5410b3a30a99-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7f4736b52b7063d2389b5410b3a30a99 columnFamilyName cf2 2023-02-09 14:55:37,257 INFO [StoreOpener-7f4736b52b7063d2389b5410b3a30a99-1] regionserver.HStore(310): Store=7f4736b52b7063d2389b5410b3a30a99/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:37,258 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99 2023-02-09 14:55:37,258 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99 2023-02-09 14:55:37,260 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:55:37,261 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 7f4736b52b7063d2389b5410b3a30a99 2023-02-09 14:55:37,262 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:55:37,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 7f4736b52b7063d2389b5410b3a30a99; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59682941, jitterRate=-0.11065487563610077}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:55:37,263 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 7f4736b52b7063d2389b5410b3a30a99: Running coprocessor pre-open hook at 1675954537252Writing region info on filesystem at 1675954537252Initializing all the Stores at 1675954537253 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954537253Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954537253Cleaning up temporary data from old regions at 1675954537259 (+6 ms)Cleaning up detritus from prior splits at 1675954537259Running coprocessor post-open hooks at 1675954537263 (+4 ms)Region opened successfully at 1675954537263 2023-02-09 14:55:37,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99., pid=104, masterSystemTime=1675954537249 2023-02-09 14:55:37,265 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. 2023-02-09 14:55:37,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. 2023-02-09 14:55:37,265 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=7f4736b52b7063d2389b5410b3a30a99, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,265 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675954537265"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954537265"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954537265"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954537265"}]},"ts":"1675954537265"} 2023-02-09 14:55:37,268 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-02-09 14:55:37,268 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure 7f4736b52b7063d2389b5410b3a30a99, server=jenkins-hbase4.apache.org,33423,1675954443133 in 169 msec 2023-02-09 14:55:37,269 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-02-09 14:55:37,270 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=7f4736b52b7063d2389b5410b3a30a99, ASSIGN in 326 msec 2023-02-09 14:55:37,270 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 14:55:37,270 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954537270"}]},"ts":"1675954537270"} 2023-02-09 14:55:37,271 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-02-09 14:55:37,274 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 14:55:37,275 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 368 msec 2023-02-09 14:55:37,413 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:37,413 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:37,414 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:37,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-09 14:55:37,416 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-09 14:55:37,416 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:37,416 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-09 14:55:37,416 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-09 14:55:37,416 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,416 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,417 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:37,417 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:37,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:37,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:37,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:37,417 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,417 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:37,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:37,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:37,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:37,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,418 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,418 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:37,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,418 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,418 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,419 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:37,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,419 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 14:55:37,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:37,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,419 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 14:55:37,419 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 14:55:37,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:37,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,419 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 14:55:37,419 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 14:55:37,419 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 14:55:37,419 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 14:55:37,419 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:37,419 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:37,419 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 14:55:37,419 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 14:55:37,419 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 14:55:37,419 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 14:55:37,419 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 14:55:37,419 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 14:55:37,419 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 14:55:37,419 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 14:55:37,419 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:37,419 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 14:55:37,420 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:37,420 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:37,420 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. 2023-02-09 14:55:37,420 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. started... 2023-02-09 14:55:37,421 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7f4736b52b7063d2389b5410b3a30a99 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 14:55:37,421 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,421 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,421 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,421 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:37,421 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,421 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:37,433 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf1/53f2107571914f01a426c16d28da71d9 2023-02-09 14:55:37,444 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf2/05cf5e7a46ab48ccb150a7fa3fe763d4 2023-02-09 14:55:37,448 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf1/53f2107571914f01a426c16d28da71d9 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/53f2107571914f01a426c16d28da71d9 2023-02-09 14:55:37,451 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/53f2107571914f01a426c16d28da71d9, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 14:55:37,452 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf2/05cf5e7a46ab48ccb150a7fa3fe763d4 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/05cf5e7a46ab48ccb150a7fa3fe763d4 2023-02-09 14:55:37,455 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/05cf5e7a46ab48ccb150a7fa3fe763d4, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 14:55:37,456 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 7f4736b52b7063d2389b5410b3a30a99 in 35ms, sequenceid=6, compaction requested=false 2023-02-09 14:55:37,456 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-02-09 14:55:37,456 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7f4736b52b7063d2389b5410b3a30a99: 2023-02-09 14:55:37,456 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. 2023-02-09 14:55:37,456 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:37,456 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:37,456 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,457 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 14:55:37,457 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 14:55:37,458 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,458 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:37,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:37,458 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,458 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:37,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:37,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:37,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:37,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:37,460 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-09 14:55:37,460 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4ee0a0af[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:37,460 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-09 14:55:37,460 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,462 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,462 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,462 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,462 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 14:55:37,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,462 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 14:55:37,462 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,462 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:37,462 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:55:37,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,462 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 14:55:37,462 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 14:55:37,462 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 14:55:37,462 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 14:55:37,462 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:55:37,462 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:55:37,463 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 14:55:37,463 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:37,464 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 14:55:37,464 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 14:55:37,464 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 14:55:37,465 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:37,464 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,465 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 14:55:37,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:37,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:37,464 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:37,465 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 14:55:37,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:37,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:37,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:37,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:37,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:37,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,33423,1675954443133': 2023-02-09 14:55:37,467 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-09 14:55:37,468 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-09 14:55:37,468 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:37,468 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:37,468 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-09 14:55:37,468 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:37,469 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,469 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,469 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,469 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,469 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:37,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:37,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:37,469 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:37,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,470 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,469 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:37,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:37,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:37,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:37,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:37,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:37,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:37,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:37,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:37,470 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:37,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,471 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:37,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:37,472 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:37,472 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,472 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,476 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,476 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:37,476 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:37,476 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:37,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:37,476 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-09 14:55:37,476 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:37,476 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:37,476 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:37,476 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:37,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:37,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:37,476 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-09 14:55:37,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:37,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:37,476 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:37,476 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:37,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:37,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:37,476 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:37,476 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:37,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:37,477 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,477 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,477 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:37,477 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:37,477 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,477 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,477 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,477 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,577 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:37,577 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:37,577 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:37,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:37,587 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:37,587 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:37,588 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:37,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:37,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-09 14:55:37,590 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-09 14:55:37,590 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:37,590 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-09 14:55:37,590 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-09 14:55:37,590 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,590 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,591 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:37,591 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:37,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:37,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:37,591 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:37,591 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:37,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:37,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:37,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:37,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,592 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,592 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:37,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,592 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,592 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,592 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:37,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:37,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,592 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 14:55:37,592 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 14:55:37,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:37,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,593 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 14:55:37,593 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 14:55:37,593 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 14:55:37,593 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 14:55:37,593 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 14:55:37,593 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 14:55:37,593 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:37,593 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 14:55:37,593 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 14:55:37,593 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 14:55:37,593 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 14:55:37,593 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 14:55:37,593 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 14:55:37,593 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:37,593 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 14:55:37,593 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 14:55:37,593 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 14:55:37,593 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 14:55:37,593 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:37,593 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 14:55:37,594 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:37,594 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:37,594 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. 2023-02-09 14:55:37,594 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. started... 2023-02-09 14:55:37,594 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7f4736b52b7063d2389b5410b3a30a99 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:55:37,594 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,602 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,602 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,602 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:37,602 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,603 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:37,605 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf1/ab66a158e33c4c48be0a4f5506d6562d 2023-02-09 14:55:37,615 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf2/206c1030856d4037bb91f74f1fd95793 2023-02-09 14:55:37,619 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf1/ab66a158e33c4c48be0a4f5506d6562d as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/ab66a158e33c4c48be0a4f5506d6562d 2023-02-09 14:55:37,624 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/ab66a158e33c4c48be0a4f5506d6562d, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 14:55:37,625 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf2/206c1030856d4037bb91f74f1fd95793 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/206c1030856d4037bb91f74f1fd95793 2023-02-09 14:55:37,629 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/206c1030856d4037bb91f74f1fd95793, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 14:55:37,630 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7f4736b52b7063d2389b5410b3a30a99 in 36ms, sequenceid=19, compaction requested=false 2023-02-09 14:55:37,630 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7f4736b52b7063d2389b5410b3a30a99: 2023-02-09 14:55:37,630 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. 2023-02-09 14:55:37,630 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:37,630 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:37,630 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,630 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 14:55:37,630 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 14:55:37,631 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:37,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:37,631 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:37,632 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,632 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:37,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:37,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:37,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:37,633 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-09 14:55:37,633 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@686efc2f[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:37,634 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-09 14:55:37,634 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,635 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,635 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,635 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,635 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 14:55:37,635 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,635 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:37,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,635 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 14:55:37,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,635 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 14:55:37,635 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 14:55:37,635 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 14:55:37,635 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:55:37,635 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:55:37,635 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 14:55:37,635 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:55:37,637 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,637 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 14:55:37,637 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:37,637 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 14:55:37,637 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:37,637 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 14:55:37,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,637 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 14:55:37,637 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 14:55:37,637 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:37,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:37,637 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 14:55:37,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:37,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:37,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:37,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:37,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:37,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:37,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,33423,1675954443133': 2023-02-09 14:55:37,645 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-09 14:55:37,645 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-09 14:55:37,645 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:37,645 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:37,645 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-09 14:55:37,645 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:37,648 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,648 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,648 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,648 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:37,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:37,648 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:37,648 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:37,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:37,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:37,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:37,648 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:37,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:37,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,649 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:37,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:37,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:37,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:37,650 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:37,650 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:37,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:37,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:37,652 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,652 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,653 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,655 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:37,655 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:37,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:37,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:37,655 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:37,655 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:37,655 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:37,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:37,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:37,655 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:37,655 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:37,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:37,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:37,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:37,656 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,656 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:37,655 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-09 14:55:37,656 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:37,656 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,656 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,656 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,656 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,656 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,656 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,656 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,656 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,656 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,656 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-09 14:55:37,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:37,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:37,656 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:37,657 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:37,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:37,757 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:37,757 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:37,757 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:37,758 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:37,769 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:37,769 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:37,770 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:37,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:37,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-09 14:55:37,771 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-09 14:55:37,771 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:37,771 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-09 14:55:37,771 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-09 14:55:37,771 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,771 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,772 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,772 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:37,772 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:37,772 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:37,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:37,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:37,773 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,773 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:37,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:37,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:37,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:37,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:37,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,773 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,774 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,774 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:37,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:37,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,774 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 14:55:37,774 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 14:55:37,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:37,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,774 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 14:55:37,774 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 14:55:37,774 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 14:55:37,774 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 14:55:37,774 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:37,774 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 14:55:37,775 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 14:55:37,775 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 14:55:37,776 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,776 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,776 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:37,776 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 14:55:37,776 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 14:55:37,776 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 14:55:37,776 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 14:55:37,776 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:37,776 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 14:55:37,777 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 14:55:37,777 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 14:55:37,777 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:37,777 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 14:55:37,777 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 14:55:37,777 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 14:55:37,779 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,780 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,780 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:37,782 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:37,782 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:37,782 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. 2023-02-09 14:55:37,782 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. started... 2023-02-09 14:55:37,782 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7f4736b52b7063d2389b5410b3a30a99 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:55:37,791 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf1/2888820565e24ef0958aa78bd4e80ae6 2023-02-09 14:55:37,805 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf2/e7864f5b5a8c4a7bb77474b5ddc78609 2023-02-09 14:55:37,808 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf1/2888820565e24ef0958aa78bd4e80ae6 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/2888820565e24ef0958aa78bd4e80ae6 2023-02-09 14:55:37,811 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/2888820565e24ef0958aa78bd4e80ae6, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 14:55:37,811 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf2/e7864f5b5a8c4a7bb77474b5ddc78609 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/e7864f5b5a8c4a7bb77474b5ddc78609 2023-02-09 14:55:37,814 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/e7864f5b5a8c4a7bb77474b5ddc78609, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 14:55:37,815 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7f4736b52b7063d2389b5410b3a30a99 in 33ms, sequenceid=32, compaction requested=true 2023-02-09 14:55:37,815 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7f4736b52b7063d2389b5410b3a30a99: 2023-02-09 14:55:37,815 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. 2023-02-09 14:55:37,815 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:37,815 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:37,815 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,815 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 14:55:37,815 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 14:55:37,816 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,816 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:37,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:37,817 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,817 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:37,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:37,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:37,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:37,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:37,818 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-09 14:55:37,818 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@461f8a99[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:37,818 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-09 14:55:37,819 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,820 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,820 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,820 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,820 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 14:55:37,820 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 14:55:37,820 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,820 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:37,820 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:55:37,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,820 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 14:55:37,820 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 14:55:37,820 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 14:55:37,820 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:55:37,820 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 14:55:37,820 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:55:37,821 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 14:55:37,821 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:37,821 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 14:55:37,823 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,823 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 14:55:37,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:37,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:37,823 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 14:55:37,823 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:37,823 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:37,823 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 14:55:37,823 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 14:55:37,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:37,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:37,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:37,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:37,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:37,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,33423,1675954443133': 2023-02-09 14:55:37,826 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-09 14:55:37,826 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-09 14:55:37,826 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:37,826 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:37,826 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-09 14:55:37,826 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:37,828 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,828 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,828 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:37,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:37,828 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,828 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:37,828 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:37,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,828 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:37,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:37,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:37,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:37,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:37,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:37,828 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:37,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:37,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:37,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,829 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:37,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,829 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:37,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:37,831 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:37,831 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,831 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,841 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,841 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,841 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:37,841 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:37,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:37,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:37,841 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-09 14:55:37,841 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:37,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:37,841 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:37,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:37,841 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:37,841 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:37,841 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,841 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:37,842 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,842 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:37,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:37,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:37,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:37,842 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-09 14:55:37,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:37,842 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:37,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:37,842 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:37,842 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:37,842 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,842 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,842 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,843 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:37,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:37,843 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,942 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:37,943 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:37,943 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:37,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:37,955 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:37,955 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:37,956 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:37,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:37,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-09 14:55:37,958 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-09 14:55:37,958 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:37,958 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-09 14:55:37,958 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-09 14:55:37,958 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,958 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,961 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,961 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:37,961 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:37,961 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:37,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:37,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:37,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:37,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:37,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:37,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:37,961 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,961 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:37,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,962 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,962 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,962 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:37,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:37,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:37,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,962 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 14:55:37,962 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 14:55:37,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:37,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:37,962 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 14:55:37,962 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 14:55:37,962 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 14:55:37,962 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 14:55:37,962 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 14:55:37,962 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:37,962 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 14:55:37,962 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 14:55:37,963 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 14:55:37,963 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 14:55:37,963 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:37,962 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 14:55:37,963 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 14:55:37,963 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 14:55:37,963 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 14:55:37,963 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 14:55:37,963 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:37,963 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:37,963 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. 2023-02-09 14:55:37,963 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:37,963 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. started... 2023-02-09 14:55:37,963 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 14:55:37,963 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 14:55:37,963 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 14:55:37,963 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7f4736b52b7063d2389b5410b3a30a99 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:55:37,964 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,965 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,965 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,965 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:37,965 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,965 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:37,972 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf1/ce71e105a81c4fa2bbf935982462b195 2023-02-09 14:55:37,985 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf2/8cb5ab459fc844c196ea77bb2d5bbc92 2023-02-09 14:55:37,989 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf1/ce71e105a81c4fa2bbf935982462b195 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/ce71e105a81c4fa2bbf935982462b195 2023-02-09 14:55:37,992 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/ce71e105a81c4fa2bbf935982462b195, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 14:55:37,992 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf2/8cb5ab459fc844c196ea77bb2d5bbc92 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/8cb5ab459fc844c196ea77bb2d5bbc92 2023-02-09 14:55:37,996 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/8cb5ab459fc844c196ea77bb2d5bbc92, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 14:55:37,996 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7f4736b52b7063d2389b5410b3a30a99 in 33ms, sequenceid=45, compaction requested=true 2023-02-09 14:55:37,996 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7f4736b52b7063d2389b5410b3a30a99: 2023-02-09 14:55:37,996 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. 2023-02-09 14:55:37,997 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:37,997 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:37,997 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,997 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 14:55:37,997 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 14:55:37,998 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:37,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:37,998 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:37,998 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:37,998 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:37,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:37,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:37,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:37,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:37,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:38,000 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-09 14:55:38,000 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5803be01[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:38,000 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-09 14:55:38,000 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,003 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,003 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,003 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,003 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,003 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:38,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,003 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 14:55:38,003 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 14:55:38,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,003 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 14:55:38,003 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 14:55:38,003 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:55:38,003 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:55:38,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,003 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 14:55:38,003 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 14:55:38,003 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:55:38,004 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 14:55:38,004 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:38,004 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 14:55:38,005 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 14:55:38,005 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:38,005 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 14:55:38,009 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:38,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:38,010 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 14:55:38,010 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:38,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:38,010 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 14:55:38,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:38,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:38,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:38,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:38,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:38,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:38,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,33423,1675954443133': 2023-02-09 14:55:38,013 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-09 14:55:38,013 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-09 14:55:38,013 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:38,013 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:38,013 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-09 14:55:38,013 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:38,015 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,015 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:38,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:38,015 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,015 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:38,015 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,015 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:38,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:38,015 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:38,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,016 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:38,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:38,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:38,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:38,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:38,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:38,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:38,016 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:38,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:38,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:38,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,017 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:38,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:38,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:38,018 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:38,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:38,019 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,019 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,021 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,021 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:38,021 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:38,021 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:38,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:38,022 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:38,022 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:38,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:38,022 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:38,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:38,022 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,021 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:38,022 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:38,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:38,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:38,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:38,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:38,022 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-09 14:55:38,022 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:38,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:38,022 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:38,022 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:38,022 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,022 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:38,022 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-09 14:55:38,022 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:38,023 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:38,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:38,023 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,023 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:38,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:38,023 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,123 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:38,123 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:38,123 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:38,124 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:38,135 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:38,135 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:38,136 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:38,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:38,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-09 14:55:38,137 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-09 14:55:38,137 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:38,137 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-09 14:55:38,137 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-09 14:55:38,138 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,138 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:38,139 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,139 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:38,139 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:38,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:38,139 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:38,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:38,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:38,139 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,139 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:38,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:38,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:38,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:38,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:38,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:38,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:38,139 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,140 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,140 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:38,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:38,140 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 14:55:38,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:38,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:38,140 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 14:55:38,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:38,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:38,140 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 14:55:38,140 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 14:55:38,140 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 14:55:38,140 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 14:55:38,140 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 14:55:38,140 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 14:55:38,140 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 14:55:38,141 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 14:55:38,141 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:38,141 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 14:55:38,141 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 14:55:38,141 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 14:55:38,141 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 14:55:38,140 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:38,141 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:38,141 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 14:55:38,141 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 14:55:38,141 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 14:55:38,141 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 14:55:38,141 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 14:55:38,141 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:38,141 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:38,141 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. 2023-02-09 14:55:38,141 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. started... 2023-02-09 14:55:38,141 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7f4736b52b7063d2389b5410b3a30a99 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:55:38,142 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,144 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,144 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:38,144 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,144 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,144 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:38,149 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf1/6518997c416943cfa4e8ea893bcb7446 2023-02-09 14:55:38,159 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf2/654160d3465d4b958d6f265b4743c81d 2023-02-09 14:55:38,162 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf1/6518997c416943cfa4e8ea893bcb7446 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/6518997c416943cfa4e8ea893bcb7446 2023-02-09 14:55:38,166 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/6518997c416943cfa4e8ea893bcb7446, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 14:55:38,166 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf2/654160d3465d4b958d6f265b4743c81d as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/654160d3465d4b958d6f265b4743c81d 2023-02-09 14:55:38,169 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/654160d3465d4b958d6f265b4743c81d, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 14:55:38,170 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7f4736b52b7063d2389b5410b3a30a99 in 29ms, sequenceid=58, compaction requested=true 2023-02-09 14:55:38,170 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7f4736b52b7063d2389b5410b3a30a99: 2023-02-09 14:55:38,170 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. 2023-02-09 14:55:38,170 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:38,170 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:38,170 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,170 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 14:55:38,170 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 14:55:38,172 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,172 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:38,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:38,172 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,172 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:38,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:38,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:38,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:38,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:38,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:38,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-09 14:55:38,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@13ad9876[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:38,174 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-09 14:55:38,174 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,175 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,175 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,175 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 14:55:38,175 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 14:55:38,175 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 14:55:38,175 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 14:55:38,175 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:55:38,175 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,175 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:55:38,175 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,175 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:38,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,175 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 14:55:38,175 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 14:55:38,175 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:55:38,179 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 14:55:38,180 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:38,180 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 14:55:38,183 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,183 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 14:55:38,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:38,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:38,183 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:38,183 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 14:55:38,183 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 14:55:38,183 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:38,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:38,183 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 14:55:38,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:38,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:38,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:38,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:38,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:38,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:38,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,33423,1675954443133': 2023-02-09 14:55:38,186 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-09 14:55:38,186 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-09 14:55:38,186 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:38,186 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:38,186 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-09 14:55:38,186 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:38,187 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,187 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,187 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,188 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:38,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:38,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:38,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,187 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:38,187 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,188 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:38,188 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:38,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:38,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:38,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:38,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:38,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:38,188 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:38,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:38,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:38,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,189 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:38,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:38,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:38,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,190 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:38,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:38,191 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:38,191 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:38,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,194 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,194 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:38,194 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:38,194 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:38,194 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:38,194 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:38,194 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:38,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:38,195 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-09 14:55:38,195 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:38,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:38,194 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:38,194 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:38,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:38,194 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:38,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:38,195 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-09 14:55:38,194 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:38,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:38,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:38,195 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:38,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:38,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:38,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:38,195 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:38,195 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,195 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,195 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,195 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:38,195 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,195 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:38,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:38,295 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:38,295 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:38,295 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:38,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:38,306 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:38,306 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:38,307 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:38,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:38,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-09 14:55:38,308 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-09 14:55:38,308 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:38,308 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-09 14:55:38,308 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-09 14:55:38,309 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,309 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:38,310 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:38,310 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:38,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:38,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:38,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:38,310 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:38,310 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:38,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:38,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:38,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:38,310 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,310 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:38,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:38,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:38,311 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,311 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,311 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:38,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:38,311 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 14:55:38,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:38,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:38,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:38,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:38,311 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 14:55:38,311 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 14:55:38,311 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-09 14:55:38,311 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 14:55:38,311 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 14:55:38,311 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:38,311 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 14:55:38,311 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 14:55:38,311 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 14:55:38,311 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 14:55:38,311 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 14:55:38,311 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 14:55:38,312 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:38,311 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-09 14:55:38,312 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 14:55:38,312 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-09 14:55:38,312 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-09 14:55:38,312 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:38,312 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:38,312 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:38,312 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-09 14:55:38,312 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 14:55:38,312 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. 2023-02-09 14:55:38,312 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 14:55:38,312 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. started... 2023-02-09 14:55:38,312 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7f4736b52b7063d2389b5410b3a30a99 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:55:38,313 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,314 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,314 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,314 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:38,314 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,314 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:38,320 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf1/5df939acb08e4980932e6636e8718416 2023-02-09 14:55:38,330 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf2/1482100fb833401998e6cdff4eaa9318 2023-02-09 14:55:38,334 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf1/5df939acb08e4980932e6636e8718416 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/5df939acb08e4980932e6636e8718416 2023-02-09 14:55:38,337 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/5df939acb08e4980932e6636e8718416, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 14:55:38,338 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf2/1482100fb833401998e6cdff4eaa9318 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/1482100fb833401998e6cdff4eaa9318 2023-02-09 14:55:38,341 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/1482100fb833401998e6cdff4eaa9318, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 14:55:38,342 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7f4736b52b7063d2389b5410b3a30a99 in 30ms, sequenceid=71, compaction requested=true 2023-02-09 14:55:38,342 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7f4736b52b7063d2389b5410b3a30a99: 2023-02-09 14:55:38,342 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. 2023-02-09 14:55:38,342 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:38,342 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:38,342 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,342 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-09 14:55:38,342 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-09 14:55:38,344 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:38,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:38,344 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:38,345 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,345 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:38,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:38,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:38,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:38,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:38,346 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-09 14:55:38,346 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c3d81a8[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:38,346 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-09 14:55:38,346 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,347 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,347 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,347 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,347 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,347 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:38,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,347 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 14:55:38,348 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 14:55:38,348 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 14:55:38,348 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 14:55:38,348 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:55:38,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,348 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:55:38,348 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-09 14:55:38,348 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-09 14:55:38,348 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:55:38,349 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 14:55:38,349 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:38,349 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 14:55:38,350 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 14:55:38,350 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:38,350 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-09 14:55:38,350 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 14:55:38,350 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,350 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:38,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:38,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:38,350 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-09 14:55:38,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:38,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:38,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:38,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:38,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:38,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:38,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:38,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,33423,1675954443133': 2023-02-09 14:55:38,353 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-09 14:55:38,353 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-09 14:55:38,353 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:38,353 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:38,353 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-09 14:55:38,353 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:38,357 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,357 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,357 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,357 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:38,357 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:38,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:38,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:38,357 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,357 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:38,357 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:38,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:38,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:38,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:38,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:38,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:38,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:38,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:38,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,358 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:38,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,358 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:38,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:38,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:38,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,359 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:38,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:38,360 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-09 14:55:38,360 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:38,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,362 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,362 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:38,362 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:38,362 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:38,362 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:38,363 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-09 14:55:38,363 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:38,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:38,363 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:38,362 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:38,362 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:38,363 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:38,362 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:38,363 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-09 14:55:38,363 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:38,363 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:38,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:38,363 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-09 14:55:38,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:38,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:38,363 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:38,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:38,363 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,363 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,363 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:38,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:38,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:38,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:38,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:38,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:38,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:38,363 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-09 14:55:38,364 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-09 14:55:38,463 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:38,463 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:38,464 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:38,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:38,471 INFO [Listener at localhost/40823] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,472 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33423] regionserver.StoreUtils(129): cannot split hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/ab66a158e33c4c48be0a4f5506d6562d because midkey is the same as first or last row 2023-02-09 14:55:38,472 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33423] regionserver.StoreUtils(129): cannot split hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/206c1030856d4037bb91f74f1fd95793 because midkey is the same as first or last row 2023-02-09 14:55:38,473 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33423] regionserver.HRegion(2744): Flushing 7f4736b52b7063d2389b5410b3a30a99 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 14:55:38,483 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33423] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf1/b2201cca2bee4359992743e686431eb5 2023-02-09 14:55:38,497 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33423] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf2/0c81d9f086b54e9c8911ede2eb32277c 2023-02-09 14:55:38,502 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33423] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf1/b2201cca2bee4359992743e686431eb5 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/b2201cca2bee4359992743e686431eb5 2023-02-09 14:55:38,506 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33423] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/b2201cca2bee4359992743e686431eb5, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 14:55:38,506 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33423] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/.tmp/cf2/0c81d9f086b54e9c8911ede2eb32277c as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/0c81d9f086b54e9c8911ede2eb32277c 2023-02-09 14:55:38,511 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33423] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/0c81d9f086b54e9c8911ede2eb32277c, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 14:55:38,511 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33423] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 7f4736b52b7063d2389b5410b3a30a99 in 38ms, sequenceid=82, compaction requested=true 2023-02-09 14:55:38,512 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33423] regionserver.HRegion(2445): Flush status journal for 7f4736b52b7063d2389b5410b3a30a99: 2023-02-09 14:55:38,513 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33423] regionserver.StoreUtils(129): cannot split hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/ab66a158e33c4c48be0a4f5506d6562d because midkey is the same as first or last row 2023-02-09 14:55:38,513 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=33423] regionserver.StoreUtils(129): cannot split hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/206c1030856d4037bb91f74f1fd95793 because midkey is the same as first or last row 2023-02-09 14:55:38,513 DEBUG [Listener at localhost/40823] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,513 DEBUG [Listener at localhost/40823] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-02-09 14:55:38,513 WARN [Listener at localhost/40823] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 14:55:38,513 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:38,514 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:38,515 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:38,515 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:38,515 INFO [Listener at localhost/40823] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-02-09 14:55:38,516 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testInvalidSplitKey 2023-02-09 14:55:38,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:38,518 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:38,518 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:38,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 14:55:38,519 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954538519"}]},"ts":"1675954538519"} 2023-02-09 14:55:38,520 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-02-09 14:55:38,522 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-02-09 14:55:38,522 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=7f4736b52b7063d2389b5410b3a30a99, UNASSIGN}] 2023-02-09 14:55:38,523 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=7f4736b52b7063d2389b5410b3a30a99, UNASSIGN 2023-02-09 14:55:38,524 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=7f4736b52b7063d2389b5410b3a30a99, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:38,525 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675954538524"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954538524"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954538524"}]},"ts":"1675954538524"} 2023-02-09 14:55:38,526 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure 7f4736b52b7063d2389b5410b3a30a99, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:55:38,619 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:38,619 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:38,620 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 14:55:38,677 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 7f4736b52b7063d2389b5410b3a30a99 2023-02-09 14:55:38,677 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 7f4736b52b7063d2389b5410b3a30a99, disabling compactions & flushes 2023-02-09 14:55:38,677 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. 2023-02-09 14:55:38,677 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. 2023-02-09 14:55:38,677 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. after waiting 0 ms 2023-02-09 14:55:38,677 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. 2023-02-09 14:55:38,681 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 14:55:38,681 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:55:38,682 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99. 2023-02-09 14:55:38,682 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 7f4736b52b7063d2389b5410b3a30a99: Waiting for close lock at 1675954538677Running coprocessor pre-close hooks at 1675954538677Disabling compacts and flushes for region at 1675954538677Disabling writes for close at 1675954538677Writing region close event to WAL at 1675954538678 (+1 ms)Running coprocessor post-close hooks at 1675954538681 (+3 ms)Closed at 1675954538682 (+1 ms) 2023-02-09 14:55:38,683 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 7f4736b52b7063d2389b5410b3a30a99 2023-02-09 14:55:38,683 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=7f4736b52b7063d2389b5410b3a30a99, regionState=CLOSED 2023-02-09 14:55:38,683 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1675954538683"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954538683"}]},"ts":"1675954538683"} 2023-02-09 14:55:38,686 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-02-09 14:55:38,686 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure 7f4736b52b7063d2389b5410b3a30a99, server=jenkins-hbase4.apache.org,33423,1675954443133 in 158 msec 2023-02-09 14:55:38,687 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-02-09 14:55:38,687 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=7f4736b52b7063d2389b5410b3a30a99, UNASSIGN in 164 msec 2023-02-09 14:55:38,688 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954538688"}]},"ts":"1675954538688"} 2023-02-09 14:55:38,689 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-02-09 14:55:38,691 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-02-09 14:55:38,692 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 175 msec 2023-02-09 14:55:38,820 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:38,820 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:38,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-09 14:55:38,821 INFO [Listener at localhost/40823] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-02-09 14:55:38,821 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:38,822 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:38,822 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testInvalidSplitKey 2023-02-09 14:55:38,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:38,824 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:38,825 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:38,825 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:38,825 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:38,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-09 14:55:38,828 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99 2023-02-09 14:55:38,829 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/recovered.edits] 2023-02-09 14:55:38,832 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/2888820565e24ef0958aa78bd4e80ae6 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/2888820565e24ef0958aa78bd4e80ae6 2023-02-09 14:55:38,833 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/53f2107571914f01a426c16d28da71d9 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/53f2107571914f01a426c16d28da71d9 2023-02-09 14:55:38,834 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/5df939acb08e4980932e6636e8718416 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/5df939acb08e4980932e6636e8718416 2023-02-09 14:55:38,835 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/6518997c416943cfa4e8ea893bcb7446 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/6518997c416943cfa4e8ea893bcb7446 2023-02-09 14:55:38,836 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/ab66a158e33c4c48be0a4f5506d6562d to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/ab66a158e33c4c48be0a4f5506d6562d 2023-02-09 14:55:38,837 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/b2201cca2bee4359992743e686431eb5 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/b2201cca2bee4359992743e686431eb5 2023-02-09 14:55:38,838 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/ce71e105a81c4fa2bbf935982462b195 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf1/ce71e105a81c4fa2bbf935982462b195 2023-02-09 14:55:38,840 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/05cf5e7a46ab48ccb150a7fa3fe763d4 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/05cf5e7a46ab48ccb150a7fa3fe763d4 2023-02-09 14:55:38,841 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/0c81d9f086b54e9c8911ede2eb32277c to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/0c81d9f086b54e9c8911ede2eb32277c 2023-02-09 14:55:38,842 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/1482100fb833401998e6cdff4eaa9318 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/1482100fb833401998e6cdff4eaa9318 2023-02-09 14:55:38,843 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/206c1030856d4037bb91f74f1fd95793 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/206c1030856d4037bb91f74f1fd95793 2023-02-09 14:55:38,844 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/654160d3465d4b958d6f265b4743c81d to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/654160d3465d4b958d6f265b4743c81d 2023-02-09 14:55:38,845 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/8cb5ab459fc844c196ea77bb2d5bbc92 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/8cb5ab459fc844c196ea77bb2d5bbc92 2023-02-09 14:55:38,846 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/e7864f5b5a8c4a7bb77474b5ddc78609 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/cf2/e7864f5b5a8c4a7bb77474b5ddc78609 2023-02-09 14:55:38,848 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/recovered.edits/85.seqid to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99/recovered.edits/85.seqid 2023-02-09 14:55:38,849 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testInvalidSplitKey/7f4736b52b7063d2389b5410b3a30a99 2023-02-09 14:55:38,849 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-09 14:55:38,850 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:38,852 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-02-09 14:55:38,855 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-02-09 14:55:38,856 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:38,856 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-02-09 14:55:38,856 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954538856"}]},"ts":"9223372036854775807"} 2023-02-09 14:55:38,857 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-09 14:55:38,857 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 7f4736b52b7063d2389b5410b3a30a99, NAME => 'testInvalidSplitKey,,1675954536906.7f4736b52b7063d2389b5410b3a30a99.', STARTKEY => '', ENDKEY => ''}] 2023-02-09 14:55:38,857 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-02-09 14:55:38,857 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675954538857"}]},"ts":"9223372036854775807"} 2023-02-09 14:55:38,858 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-02-09 14:55:38,859 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:38,860 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 37 msec 2023-02-09 14:55:38,925 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:38,926 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:38,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-09 14:55:38,927 INFO [Listener at localhost/40823] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-02-09 14:55:38,939 INFO [Listener at localhost/40823] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=362 (was 341) - Thread LEAK? -, OpenFileDescriptor=736 (was 652) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=91 (was 91), ProcessCount=168 (was 168), AvailableMemoryMB=2978 (was 2981) 2023-02-09 14:55:38,952 INFO [Listener at localhost/40823] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=362, OpenFileDescriptor=736, MaxFileDescriptor=60000, SystemLoadAverage=91, ProcessCount=168, AvailableMemoryMB=2977 2023-02-09 14:55:38,952 WARN [Listener at localhost/40823] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 14:55:38,952 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:38,953 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:38,954 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 14:55:38,955 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 14:55:38,956 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 14:55:38,957 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 14:55:38,958 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 14:55:38,959 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:38,960 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28 empty. 2023-02-09 14:55:38,960 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:38,960 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-09 14:55:38,968 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-09 14:55:38,969 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => a38bfbdf613fc88c3b7b74edf1f8bd28, NAME => 'testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp 2023-02-09 14:55:38,976 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:38,976 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing a38bfbdf613fc88c3b7b74edf1f8bd28, disabling compactions & flushes 2023-02-09 14:55:38,976 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. 2023-02-09 14:55:38,976 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. 2023-02-09 14:55:38,976 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. after waiting 0 ms 2023-02-09 14:55:38,976 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. 2023-02-09 14:55:38,976 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. 2023-02-09 14:55:38,976 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for a38bfbdf613fc88c3b7b74edf1f8bd28: Waiting for close lock at 1675954538976Disabling compacts and flushes for region at 1675954538976Disabling writes for close at 1675954538976Writing region close event to WAL at 1675954538976Closed at 1675954538976 2023-02-09 14:55:38,978 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 14:55:38,979 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675954538979"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954538979"}]},"ts":"1675954538979"} 2023-02-09 14:55:38,980 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 14:55:38,980 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 14:55:38,981 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954538980"}]},"ts":"1675954538980"} 2023-02-09 14:55:38,981 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-09 14:55:38,985 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 14:55:38,986 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 14:55:38,986 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 14:55:38,986 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 14:55:38,986 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 14:55:38,986 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28, ASSIGN}] 2023-02-09 14:55:38,987 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28, ASSIGN 2023-02-09 14:55:38,988 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33553,1675954443180; forceNewPlan=false, retain=false 2023-02-09 14:55:39,138 INFO [jenkins-hbase4:43829] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 14:55:39,139 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=a38bfbdf613fc88c3b7b74edf1f8bd28, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,139 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675954539139"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954539139"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954539139"}]},"ts":"1675954539139"} 2023-02-09 14:55:39,141 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure a38bfbdf613fc88c3b7b74edf1f8bd28, server=jenkins-hbase4.apache.org,33553,1675954443180}] 2023-02-09 14:55:39,293 DEBUG [RSProcedureDispatcher-pool-4] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,293 DEBUG [RSProcedureDispatcher-pool-4] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-09 14:55:39,296 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34134, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-09 14:55:39,300 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. 2023-02-09 14:55:39,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a38bfbdf613fc88c3b7b74edf1f8bd28, NAME => 'testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:55:39,301 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:55:39,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:39,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:39,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:39,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:39,303 INFO [StoreOpener-a38bfbdf613fc88c3b7b74edf1f8bd28-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:39,304 DEBUG [StoreOpener-a38bfbdf613fc88c3b7b74edf1f8bd28-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1 2023-02-09 14:55:39,304 DEBUG [StoreOpener-a38bfbdf613fc88c3b7b74edf1f8bd28-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1 2023-02-09 14:55:39,304 INFO [StoreOpener-a38bfbdf613fc88c3b7b74edf1f8bd28-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a38bfbdf613fc88c3b7b74edf1f8bd28 columnFamilyName cf1 2023-02-09 14:55:39,305 INFO [StoreOpener-a38bfbdf613fc88c3b7b74edf1f8bd28-1] regionserver.HStore(310): Store=a38bfbdf613fc88c3b7b74edf1f8bd28/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:39,305 INFO [StoreOpener-a38bfbdf613fc88c3b7b74edf1f8bd28-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:39,306 DEBUG [StoreOpener-a38bfbdf613fc88c3b7b74edf1f8bd28-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2 2023-02-09 14:55:39,306 DEBUG [StoreOpener-a38bfbdf613fc88c3b7b74edf1f8bd28-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2 2023-02-09 14:55:39,306 INFO [StoreOpener-a38bfbdf613fc88c3b7b74edf1f8bd28-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a38bfbdf613fc88c3b7b74edf1f8bd28 columnFamilyName cf2 2023-02-09 14:55:39,307 INFO [StoreOpener-a38bfbdf613fc88c3b7b74edf1f8bd28-1] regionserver.HStore(310): Store=a38bfbdf613fc88c3b7b74edf1f8bd28/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:39,307 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:39,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:39,309 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:55:39,310 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:39,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:55:39,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a38bfbdf613fc88c3b7b74edf1f8bd28; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74959644, jitterRate=0.1169857382774353}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:55:39,312 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a38bfbdf613fc88c3b7b74edf1f8bd28: Running coprocessor pre-open hook at 1675954539302Writing region info on filesystem at 1675954539302Initializing all the Stores at 1675954539302Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954539302Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954539303 (+1 ms)Cleaning up temporary data from old regions at 1675954539308 (+5 ms)Cleaning up detritus from prior splits at 1675954539309 (+1 ms)Running coprocessor post-open hooks at 1675954539312 (+3 ms)Region opened successfully at 1675954539312 2023-02-09 14:55:39,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28., pid=111, masterSystemTime=1675954539293 2023-02-09 14:55:39,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. 2023-02-09 14:55:39,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. 2023-02-09 14:55:39,316 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=a38bfbdf613fc88c3b7b74edf1f8bd28, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,316 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675954539316"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954539316"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954539316"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954539316"}]},"ts":"1675954539316"} 2023-02-09 14:55:39,319 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-02-09 14:55:39,319 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure a38bfbdf613fc88c3b7b74edf1f8bd28, server=jenkins-hbase4.apache.org,33553,1675954443180 in 176 msec 2023-02-09 14:55:39,320 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-02-09 14:55:39,320 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28, ASSIGN in 333 msec 2023-02-09 14:55:39,321 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 14:55:39,321 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954539321"}]},"ts":"1675954539321"} 2023-02-09 14:55:39,322 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-09 14:55:39,324 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 14:55:39,325 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 369 msec 2023-02-09 14:55:39,459 DEBUG [Listener at localhost/40823] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-09 14:55:39,463 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34150, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-09 14:55:39,468 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:39,468 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:39,469 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:39,470 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-09 14:55:39,471 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-09 14:55:39,471 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:39,471 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-09 14:55:39,471 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 14:55:39,471 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,471 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,472 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:39,472 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:39,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:39,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:39,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:39,472 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,472 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:39,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:39,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:39,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:39,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,473 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,473 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:39,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,473 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,473 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:39,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,473 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,473 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 14:55:39,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:39,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,474 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 14:55:39,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:39,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,474 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 14:55:39,475 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 14:55:39,475 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 14:55:39,475 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:39,475 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 14:55:39,475 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 14:55:39,475 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 14:55:39,475 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:39,475 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:55:39,475 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 14:55:39,475 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 14:55:39,475 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:55:39,475 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 14:55:39,476 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:39,476 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 14:55:39,476 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 14:55:39,476 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:55:39,476 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:39,476 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:39,476 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. 2023-02-09 14:55:39,476 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. started... 2023-02-09 14:55:39,477 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a38bfbdf613fc88c3b7b74edf1f8bd28 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 14:55:39,478 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,479 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,479 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,479 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:39,479 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,479 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:39,485 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5 2023-02-09 14:55:39,498 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf2/e59ca37300bd48b6a4ee7ac39554cbaf 2023-02-09 14:55:39,502 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5 2023-02-09 14:55:39,505 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 14:55:39,506 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf2/e59ca37300bd48b6a4ee7ac39554cbaf as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e59ca37300bd48b6a4ee7ac39554cbaf 2023-02-09 14:55:39,509 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e59ca37300bd48b6a4ee7ac39554cbaf, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 14:55:39,510 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for a38bfbdf613fc88c3b7b74edf1f8bd28 in 33ms, sequenceid=6, compaction requested=false 2023-02-09 14:55:39,510 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-02-09 14:55:39,510 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a38bfbdf613fc88c3b7b74edf1f8bd28: 2023-02-09 14:55:39,510 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. 2023-02-09 14:55:39,510 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:39,510 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:39,510 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,511 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 14:55:39,511 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 14:55:39,512 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,512 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:39,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:39,512 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,512 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:39,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:39,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:39,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:39,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:39,514 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-09 14:55:39,514 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@364c99f4[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:39,514 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 14:55:39,514 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,515 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,515 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,515 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,515 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:55:39,515 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 14:55:39,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,515 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,515 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:55:39,515 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 14:55:39,515 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:55:39,515 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:39,515 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:55:39,516 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 14:55:39,516 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:55:39,515 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:55:39,518 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 14:55:39,518 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:39,518 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 14:55:39,518 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 14:55:39,518 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,518 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 14:55:39,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,519 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:39,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:39,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:39,519 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:39,519 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 14:55:39,519 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 14:55:39,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:39,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:39,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:39,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:39,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:39,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,33553,1675954443180': 2023-02-09 14:55:39,522 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 14:55:39,522 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-09 14:55:39,522 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:39,522 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:39,522 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-09 14:55:39,522 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:39,523 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,523 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:39,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:39,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,523 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:39,523 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,523 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:39,523 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:39,523 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:39,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:39,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,524 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,523 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:39,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:39,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:39,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:39,524 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:39,524 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:39,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:39,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:39,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,526 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:39,526 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:39,526 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,529 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,529 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,529 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:39,529 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:39,529 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:39,529 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:39,529 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:39,529 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-09 14:55:39,529 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:39,529 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:39,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:39,529 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,529 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:39,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:39,529 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:39,529 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 14:55:39,529 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:39,530 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:39,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:39,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:39,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:39,529 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:39,530 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,530 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,530 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,530 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,530 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:39,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:39,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:39,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:39,630 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:39,630 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:39,630 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:39,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:39,641 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:39,641 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:39,641 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:39,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:39,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-09 14:55:39,643 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-09 14:55:39,643 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:39,643 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-09 14:55:39,643 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 14:55:39,643 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,643 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,645 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:39,645 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:39,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:39,645 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:39,645 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:39,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:39,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:39,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:39,645 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:39,645 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:39,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,646 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,646 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,646 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:39,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,646 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 14:55:39,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:39,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:39,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,647 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 14:55:39,647 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 14:55:39,646 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 14:55:39,647 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 14:55:39,647 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 14:55:39,647 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:39,647 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 14:55:39,647 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:55:39,647 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 14:55:39,647 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 14:55:39,647 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 14:55:39,647 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 14:55:39,647 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:39,647 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:39,647 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:39,647 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:55:39,647 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 14:55:39,647 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 14:55:39,647 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:39,647 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. 2023-02-09 14:55:39,647 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:55:39,647 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 14:55:39,647 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 14:55:39,647 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. started... 2023-02-09 14:55:39,647 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 14:55:39,647 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a38bfbdf613fc88c3b7b74edf1f8bd28 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:55:39,649 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,649 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,649 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,649 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:39,649 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,649 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:39,655 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf1/a551eb3bbc834e1f80951082e085d1fd 2023-02-09 14:55:39,665 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf2/d30ffcea033d4133941ec24b991abfbe 2023-02-09 14:55:39,668 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf1/a551eb3bbc834e1f80951082e085d1fd as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/a551eb3bbc834e1f80951082e085d1fd 2023-02-09 14:55:39,671 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/a551eb3bbc834e1f80951082e085d1fd, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 14:55:39,671 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf2/d30ffcea033d4133941ec24b991abfbe as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/d30ffcea033d4133941ec24b991abfbe 2023-02-09 14:55:39,675 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/d30ffcea033d4133941ec24b991abfbe, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 14:55:39,675 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a38bfbdf613fc88c3b7b74edf1f8bd28 in 28ms, sequenceid=19, compaction requested=false 2023-02-09 14:55:39,676 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a38bfbdf613fc88c3b7b74edf1f8bd28: 2023-02-09 14:55:39,676 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. 2023-02-09 14:55:39,676 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:39,676 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:39,676 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,676 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 14:55:39,676 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 14:55:39,685 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,685 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:39,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:39,685 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,685 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:39,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:39,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:39,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:39,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:39,687 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-09 14:55:39,687 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@737aca8a[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:39,687 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 14:55:39,687 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,688 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,688 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,688 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,688 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:55:39,688 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 14:55:39,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,688 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,688 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:55:39,688 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 14:55:39,688 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:55:39,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,688 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:39,688 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:55:39,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,689 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:55:39,689 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 14:55:39,689 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:55:39,690 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 14:55:39,690 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:39,690 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 14:55:39,691 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 14:55:39,691 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 14:55:39,691 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:39,691 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,691 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 14:55:39,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:39,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:39,691 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:39,692 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 14:55:39,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:39,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:39,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:39,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:39,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:39,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,33553,1675954443180': 2023-02-09 14:55:39,694 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 14:55:39,694 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-09 14:55:39,694 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:39,694 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:39,694 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-09 14:55:39,695 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:39,696 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,696 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,696 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,696 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,696 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:39,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,696 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:39,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:39,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:39,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:39,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,696 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,696 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:39,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:39,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:39,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:39,697 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:39,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:39,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:39,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,697 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:39,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:39,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:39,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,698 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,699 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:39,699 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:39,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,701 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,701 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,701 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,701 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:39,701 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,702 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,701 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:39,702 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:39,701 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:39,702 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-09 14:55:39,702 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:39,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:39,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:39,702 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:39,702 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,702 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:39,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:39,702 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 14:55:39,702 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:39,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:39,702 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:39,702 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,702 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:39,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:39,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:39,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:39,702 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,703 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,703 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:39,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:39,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:39,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:39,802 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:39,803 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:39,803 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:39,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:39,814 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:39,814 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:39,815 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:39,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:39,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-09 14:55:39,816 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-09 14:55:39,816 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:39,816 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-09 14:55:39,816 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 14:55:39,817 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,817 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,818 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:39,818 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,818 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:39,818 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:39,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:39,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:39,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:39,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:39,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:39,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:39,818 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,818 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:39,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,819 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,819 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,819 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:39,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:39,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,819 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 14:55:39,819 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 14:55:39,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:39,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,819 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 14:55:39,819 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 14:55:39,819 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 14:55:39,820 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 14:55:39,820 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 14:55:39,820 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 14:55:39,820 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 14:55:39,820 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 14:55:39,820 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 14:55:39,820 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 14:55:39,820 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:39,820 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:39,820 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:55:39,820 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:55:39,820 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 14:55:39,820 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 14:55:39,820 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 14:55:39,820 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 14:55:39,820 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:39,820 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:55:39,820 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:39,820 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:39,820 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. 2023-02-09 14:55:39,820 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. started... 2023-02-09 14:55:39,821 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a38bfbdf613fc88c3b7b74edf1f8bd28 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:55:39,822 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,822 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,823 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,823 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,823 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:39,823 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:39,832 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf1/d2e156e7ebf64b6ba510ea828773eca1 2023-02-09 14:55:39,841 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf2/333988a5c29b47449390ab71b1a3ef2f 2023-02-09 14:55:39,845 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf1/d2e156e7ebf64b6ba510ea828773eca1 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/d2e156e7ebf64b6ba510ea828773eca1 2023-02-09 14:55:39,848 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/d2e156e7ebf64b6ba510ea828773eca1, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 14:55:39,849 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf2/333988a5c29b47449390ab71b1a3ef2f as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/333988a5c29b47449390ab71b1a3ef2f 2023-02-09 14:55:39,851 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/333988a5c29b47449390ab71b1a3ef2f, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 14:55:39,852 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a38bfbdf613fc88c3b7b74edf1f8bd28 in 32ms, sequenceid=32, compaction requested=true 2023-02-09 14:55:39,852 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a38bfbdf613fc88c3b7b74edf1f8bd28: 2023-02-09 14:55:39,852 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. 2023-02-09 14:55:39,852 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:39,852 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:39,852 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,852 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 14:55:39,853 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 14:55:39,854 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,854 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:39,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:39,854 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,854 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:39,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:39,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:39,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:39,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:39,856 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-09 14:55:39,856 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@ceb82bb[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:39,856 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 14:55:39,856 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,857 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,857 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,857 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,857 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:55:39,857 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,857 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:39,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,857 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:55:39,857 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 14:55:39,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,857 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 14:55:39,857 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:55:39,857 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 14:55:39,857 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:55:39,857 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:55:39,857 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:55:39,859 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 14:55:39,859 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 14:55:39,859 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:39,859 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:39,859 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 14:55:39,859 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 14:55:39,860 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 14:55:39,860 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:39,860 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,860 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 14:55:39,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:39,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:39,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:39,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:39,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:39,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,862 ERROR [BP-2069929948-172.31.14.131-1675954438937 heartbeating to localhost/127.0.0.1:38049] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2069929948-172.31.14.131-1675954438937 (Datanode Uuid f8e8dad4-c7b5-422d-a273-bab2cf112e44) service to localhost/127.0.0.1:38049 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-09 14:55:39,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:39,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:39,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,33553,1675954443180': 2023-02-09 14:55:39,864 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 14:55:39,864 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-09 14:55:39,864 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:39,864 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:39,864 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-09 14:55:39,864 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:39,865 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,865 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,865 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,865 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:39,865 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:39,865 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:39,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,866 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,865 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:39,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:39,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:39,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:39,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:39,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:39,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:39,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:39,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:39,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,866 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:39,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,867 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:39,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:39,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,868 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:39,868 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:39,869 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,874 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:39,874 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:39,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:39,874 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:39,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:39,874 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:39,875 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:39,874 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:39,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:39,874 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:39,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:39,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:39,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:39,875 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:39,875 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,875 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,875 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:39,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:39,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:39,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:39,885 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:39,885 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,885 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:39,885 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:39,885 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-09 14:55:39,885 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:39,885 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:39,885 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,886 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,886 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 14:55:39,886 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:39,986 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:39,986 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:39,986 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:39,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:39,996 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:39,997 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:39,997 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:39,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:39,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-09 14:55:39,998 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-09 14:55:39,998 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:39,998 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-09 14:55:39,998 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 14:55:39,999 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:39,999 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,000 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,000 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:40,000 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:40,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:40,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:40,000 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:40,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:40,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:40,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:40,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:40,000 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,001 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:40,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,001 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,001 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,001 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:40,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:40,001 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 14:55:40,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:40,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,001 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 14:55:40,001 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 14:55:40,002 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 14:55:40,002 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 14:55:40,002 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 14:55:40,002 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:40,002 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 14:55:40,002 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:55:40,002 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 14:55:40,002 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 14:55:40,002 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 14:55:40,002 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 14:55:40,002 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 14:55:40,002 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:40,002 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:40,002 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:40,002 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:40,002 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:55:40,002 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 14:55:40,002 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. 2023-02-09 14:55:40,002 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 14:55:40,002 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. started... 2023-02-09 14:55:40,002 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:55:40,002 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 14:55:40,002 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 14:55:40,002 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a38bfbdf613fc88c3b7b74edf1f8bd28 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:55:40,004 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,004 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,004 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,004 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:40,004 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,004 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:40,010 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf1/5b999bec37ed41eca0c9366717969996 2023-02-09 14:55:40,020 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf2/fdc7589cbab3484eb9c2b4473a70e219 2023-02-09 14:55:40,023 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf1/5b999bec37ed41eca0c9366717969996 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/5b999bec37ed41eca0c9366717969996 2023-02-09 14:55:40,026 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/5b999bec37ed41eca0c9366717969996, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 14:55:40,027 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf2/fdc7589cbab3484eb9c2b4473a70e219 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/fdc7589cbab3484eb9c2b4473a70e219 2023-02-09 14:55:40,030 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/fdc7589cbab3484eb9c2b4473a70e219, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 14:55:40,031 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a38bfbdf613fc88c3b7b74edf1f8bd28 in 29ms, sequenceid=45, compaction requested=true 2023-02-09 14:55:40,031 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a38bfbdf613fc88c3b7b74edf1f8bd28: 2023-02-09 14:55:40,031 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. 2023-02-09 14:55:40,031 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:40,032 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:40,032 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,032 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 14:55:40,032 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 14:55:40,034 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,034 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:40,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:40,034 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,034 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:40,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:40,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:40,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:40,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:40,036 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-09 14:55:40,036 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6cbb624f[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:40,036 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 14:55:40,036 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,037 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,037 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,037 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,037 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,037 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:40,037 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:55:40,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,037 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:55:40,038 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 14:55:40,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,037 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 14:55:40,038 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:55:40,038 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:55:40,038 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 14:55:40,038 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:55:40,038 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:55:40,039 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 14:55:40,039 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,039 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:40,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:40,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:40,039 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 14:55:40,039 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 14:55:40,039 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 14:55:40,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:40,039 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:40,039 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:40,039 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 14:55:40,040 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 14:55:40,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:40,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:40,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:40,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:40,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,33553,1675954443180': 2023-02-09 14:55:40,042 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 14:55:40,042 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-09 14:55:40,042 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:40,042 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:40,042 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-09 14:55:40,042 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:40,047 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,047 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,047 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,047 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,048 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:40,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:40,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:40,047 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:40,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,048 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:40,048 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:40,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:40,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:40,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:40,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:40,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:40,048 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:40,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:40,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,049 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:40,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:40,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,050 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:40,050 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:40,051 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,053 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,053 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,053 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:40,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:40,053 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:40,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:40,053 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:40,053 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:40,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:40,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:40,053 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:40,053 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:40,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:40,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:40,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:40,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:40,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:40,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:40,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:40,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:40,056 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,056 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,056 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,056 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,056 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,056 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,056 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,056 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,056 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:40,056 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,056 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-09 14:55:40,057 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:40,057 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 14:55:40,057 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:40,157 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:40,157 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:40,157 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:40,158 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:40,168 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:40,168 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:40,169 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:40,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:40,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-09 14:55:40,171 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-09 14:55:40,171 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:40,171 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-09 14:55:40,171 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 14:55:40,171 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,171 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,172 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:40,172 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:40,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:40,172 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:40,172 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:40,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:40,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:40,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:40,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:40,173 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,173 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:40,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,173 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,173 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,173 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:40,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:40,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:40,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,174 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 14:55:40,174 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 14:55:40,174 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 14:55:40,174 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 14:55:40,174 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 14:55:40,174 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 14:55:40,174 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 14:55:40,174 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:40,174 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 14:55:40,174 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:55:40,174 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 14:55:40,174 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 14:55:40,174 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:40,174 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:40,174 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. 2023-02-09 14:55:40,174 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:40,174 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. started... 2023-02-09 14:55:40,174 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 14:55:40,174 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:55:40,174 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 14:55:40,175 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a38bfbdf613fc88c3b7b74edf1f8bd28 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:55:40,174 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 14:55:40,175 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 14:55:40,175 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:40,175 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:55:40,175 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 14:55:40,175 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 14:55:40,176 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,178 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,178 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,178 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:40,178 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,178 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:40,186 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf1/09103225a5724a4d8c9313c7f68a607e 2023-02-09 14:55:40,195 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf2/bafbe116570d4368879cf0bbc41638d3 2023-02-09 14:55:40,199 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf1/09103225a5724a4d8c9313c7f68a607e as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/09103225a5724a4d8c9313c7f68a607e 2023-02-09 14:55:40,202 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/09103225a5724a4d8c9313c7f68a607e, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 14:55:40,202 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf2/bafbe116570d4368879cf0bbc41638d3 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/bafbe116570d4368879cf0bbc41638d3 2023-02-09 14:55:40,205 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/bafbe116570d4368879cf0bbc41638d3, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 14:55:40,206 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a38bfbdf613fc88c3b7b74edf1f8bd28 in 32ms, sequenceid=58, compaction requested=true 2023-02-09 14:55:40,206 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a38bfbdf613fc88c3b7b74edf1f8bd28: 2023-02-09 14:55:40,206 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. 2023-02-09 14:55:40,206 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:40,206 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:40,206 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,206 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 14:55:40,206 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 14:55:40,210 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,210 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:40,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:40,211 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,211 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:40,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:40,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:40,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:40,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:40,212 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-09 14:55:40,212 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3eeac515[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:40,212 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 14:55:40,212 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,213 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,213 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,213 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,213 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:55:40,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,214 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:55:40,214 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 14:55:40,214 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:55:40,214 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 14:55:40,214 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:55:40,214 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,214 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:40,214 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 14:55:40,214 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:55:40,214 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:55:40,215 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 14:55:40,215 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,215 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:40,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:40,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:40,215 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 14:55:40,217 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 14:55:40,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:40,217 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:40,217 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 14:55:40,217 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:40,217 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 14:55:40,217 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 14:55:40,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:40,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:40,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:40,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:40,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,33553,1675954443180': 2023-02-09 14:55:40,219 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 14:55:40,219 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-09 14:55:40,220 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:40,220 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:40,220 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-09 14:55:40,220 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:40,221 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,221 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,221 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,221 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:40,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,221 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:40,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:40,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:40,221 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:40,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:40,221 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:40,222 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:40,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:40,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:40,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:40,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:40,222 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:40,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:40,222 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:40,224 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:40,224 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,224 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,226 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,226 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,227 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:40,227 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:40,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:40,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:40,227 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:40,227 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,227 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-09 14:55:40,227 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:40,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:40,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:40,227 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:40,227 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:40,227 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:40,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:40,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:40,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:40,227 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:40,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:40,227 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:40,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:40,227 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 14:55:40,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:40,227 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:40,228 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:40,228 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,228 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,228 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,228 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,228 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,228 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,328 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:40,328 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:40,328 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:40,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:40,339 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:40,339 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:40,340 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:40,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:40,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-09 14:55:40,341 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-09 14:55:40,341 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:40,341 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-09 14:55:40,341 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-09 14:55:40,342 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,342 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,343 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:40,343 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:40,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:40,343 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,343 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:40,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:40,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:40,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:40,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:40,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:40,343 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,343 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:40,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,344 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,344 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,344 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:40,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:40,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:40,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,344 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 14:55:40,344 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 14:55:40,344 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-09 14:55:40,344 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 14:55:40,344 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 14:55:40,344 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 14:55:40,344 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 14:55:40,344 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:40,344 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-09 14:55:40,344 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:55:40,345 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 14:55:40,345 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 14:55:40,344 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 14:55:40,345 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-09 14:55:40,345 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 14:55:40,345 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-09 14:55:40,345 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:40,345 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:40,345 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:55:40,345 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-09 14:55:40,345 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 14:55:40,345 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 14:55:40,345 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:40,345 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:40,345 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. 2023-02-09 14:55:40,345 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. started... 2023-02-09 14:55:40,345 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a38bfbdf613fc88c3b7b74edf1f8bd28 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:55:40,347 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,348 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,348 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:40,348 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,349 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,349 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:40,353 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf1/8c0ffa708ab14279b63e065ccfd3ce98 2023-02-09 14:55:40,363 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf2/e8315fec79ba4d5ab09ccd57719ed2a6 2023-02-09 14:55:40,366 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf1/8c0ffa708ab14279b63e065ccfd3ce98 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/8c0ffa708ab14279b63e065ccfd3ce98 2023-02-09 14:55:40,369 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/8c0ffa708ab14279b63e065ccfd3ce98, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 14:55:40,370 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf2/e8315fec79ba4d5ab09ccd57719ed2a6 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e8315fec79ba4d5ab09ccd57719ed2a6 2023-02-09 14:55:40,372 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e8315fec79ba4d5ab09ccd57719ed2a6, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 14:55:40,373 INFO [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for a38bfbdf613fc88c3b7b74edf1f8bd28 in 28ms, sequenceid=71, compaction requested=true 2023-02-09 14:55:40,373 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a38bfbdf613fc88c3b7b74edf1f8bd28: 2023-02-09 14:55:40,373 DEBUG [rs(jenkins-hbase4.apache.org,33553,1675954443180)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. 2023-02-09 14:55:40,373 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:40,373 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:40,373 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,373 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-09 14:55:40,373 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-09 14:55:40,375 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,375 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:40,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:40,375 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,375 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:40,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:40,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:40,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:40,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:40,377 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-09 14:55:40,377 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b6176d8[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:40,377 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-09 14:55:40,377 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,378 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,378 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,378 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,378 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:55:40,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,378 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,378 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:55:40,378 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 14:55:40,378 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:55:40,378 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-09 14:55:40,378 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 14:55:40,378 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-09 14:55:40,378 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:55:40,378 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:40,378 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:55:40,380 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 14:55:40,380 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:40,380 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,380 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 14:55:40,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:40,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:40,381 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 14:55:40,381 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:40,381 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 14:55:40,381 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-09 14:55:40,381 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:40,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:40,381 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-09 14:55:40,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:40,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:40,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:40,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:40,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,33553,1675954443180': 2023-02-09 14:55:40,384 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-09 14:55:40,384 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-09 14:55:40,384 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:40,384 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:40,384 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-09 14:55:40,384 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:40,385 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,385 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:40,385 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:40,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,385 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:40,385 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,386 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:40,385 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:40,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:40,386 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:40,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:40,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:40,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:40,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:40,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:40,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:40,386 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:40,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,387 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:40,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:40,388 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-09 14:55:40,388 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,388 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,394 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,394 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,394 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:40,394 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:40,394 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:40,394 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:40,394 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-09 14:55:40,394 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:40,394 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:40,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:40,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:40,394 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:40,394 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:40,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:40,394 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,394 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-09 14:55:40,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:40,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:40,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:40,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:40,394 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:40,394 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:40,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:40,394 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:40,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:40,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:40,395 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,395 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,395 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:40,395 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:40,395 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,395 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-09 14:55:40,495 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:40,495 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:40,495 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:40,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:40,753 WARN [Listener at localhost/40823] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-09 14:55:40,754 DEBUG [Listener at localhost/40823] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:40,755 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:40,756 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:40,756 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:40,756 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:40,756 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:55:40,756 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:40,756 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:55:40,756 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:55:41,005 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,005 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:55:41,005 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:55:41,005 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:55:41,005 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-02-09 14:55:41,005 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:55:41,005 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:55:41,006 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:55:41,006 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:55:41,006 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:55:41,007 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-09 14:55:41,008 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:55:41,008 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:55:41,008 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:55:41,008 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:55:41,008 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:41,009 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:55:41,009 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:41,009 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:41,009 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:41,009 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:55:41,009 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:41,009 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:41,010 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:41,010 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:55:41,010 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN 2023-02-09 14:55:41,010 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:41,010 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:41,010 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:41,010 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7a3a2a2c1be8b68cf6b9c5576ca1d967, daughterA=915612316d6a09fe13e83a17e2d06051, daughterB=2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:55:41,011 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:41,011 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:41,011 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:41,011 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=94ffc82cbf109e7f71870e19837a0e78, daughterA=c720e43e6c74641ba50283474b45a523, daughterB=8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:55:41,011 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:41,011 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:41,011 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:41,012 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:41,012 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41077,1675954443212, table=testRollbackForSplitTableRegionWithReplica, region=813d8870e2a4f2929f4991fa0f505d12 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=080674692710b4dfd3481fd1d48b1c60, daughterA=45103f245599b3c2b8e9be2a40c7083b, daughterB=47b175695c3d18e949263cbc873d2e48 2023-02-09 14:55:41,012 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:41,012 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:41,012 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:41,012 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:41,013 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:41,013 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 14:55:41,013 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,013 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:55:41,013 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,013 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,013 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,013 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 14:55:41,014 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:55:41,014 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:55:41,015 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954571014 2023-02-09 14:55:41,015 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:55:41,016 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,018 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:55:41,018 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=a38bfbdf613fc88c3b7b74edf1f8bd28, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33553,1675954443180, regionLocation=jenkins-hbase4.apache.org,33553,1675954443180, openSeqNum=2 2023-02-09 14:55:41,018 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:55:41,018 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954601018 2023-02-09 14:55:41,019 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954661019 2023-02-09 14:55:41,019 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 14:55:41,019 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:55:41,020 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:55:41,020 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:55:41,020 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:55:41,269 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,269 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:55:41,269 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:55:41,269 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:55:41,269 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:55:41,269 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:55:41,269 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:55:41,269 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:55:41,269 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:55:41,270 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:55:41,271 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-09 14:55:41,271 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:55:41,272 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:55:41,272 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:55:41,272 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:55:41,272 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:41,272 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:55:41,273 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:41,273 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:41,273 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:41,273 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:55:41,273 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:41,273 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:41,273 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:41,274 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:55:41,274 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN 2023-02-09 14:55:41,274 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:41,274 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:41,275 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:41,275 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7a3a2a2c1be8b68cf6b9c5576ca1d967, daughterA=915612316d6a09fe13e83a17e2d06051, daughterB=2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:55:41,275 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:41,275 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:41,276 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:41,276 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=94ffc82cbf109e7f71870e19837a0e78, daughterA=c720e43e6c74641ba50283474b45a523, daughterB=8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:55:41,276 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:41,276 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:41,276 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:41,277 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:41,277 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41077,1675954443212, table=testRollbackForSplitTableRegionWithReplica, region=813d8870e2a4f2929f4991fa0f505d12 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=080674692710b4dfd3481fd1d48b1c60, daughterA=45103f245599b3c2b8e9be2a40c7083b, daughterB=47b175695c3d18e949263cbc873d2e48 2023-02-09 14:55:41,277 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:41,277 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:41,277 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:41,277 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:41,277 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:41,278 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 14:55:41,278 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,278 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:55:41,278 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,278 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,278 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,278 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-09 14:55:41,279 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:55:41,279 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:55:41,279 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954571279 2023-02-09 14:55:41,281 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:55:41,281 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,283 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:55:41,283 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=a38bfbdf613fc88c3b7b74edf1f8bd28, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33553,1675954443180, regionLocation=jenkins-hbase4.apache.org,33553,1675954443180, openSeqNum=2 2023-02-09 14:55:41,283 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:55:41,283 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954601283 2023-02-09 14:55:41,283 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954661283 2023-02-09 14:55:41,283 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 14:55:41,283 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:55:41,285 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28, UNASSIGN}] 2023-02-09 14:55:41,285 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:55:41,285 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,285 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:55:41,285 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:55:41,533 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,534 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:55:41,534 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:55:41,534 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:55:41,534 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:55:41,534 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:55:41,534 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:55:41,534 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:55:41,534 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:55:41,535 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:55:41,536 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-09 14:55:41,536 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:55:41,537 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:55:41,537 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:55:41,537 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:55:41,537 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:41,537 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:55:41,537 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:41,537 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:41,538 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:41,538 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:55:41,538 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:41,538 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:41,538 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:41,538 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:55:41,538 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN 2023-02-09 14:55:41,539 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:41,539 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:41,539 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:41,539 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7a3a2a2c1be8b68cf6b9c5576ca1d967, daughterA=915612316d6a09fe13e83a17e2d06051, daughterB=2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:55:41,539 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:41,539 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:41,539 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:41,539 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=94ffc82cbf109e7f71870e19837a0e78, daughterA=c720e43e6c74641ba50283474b45a523, daughterB=8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:55:41,540 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:41,540 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:41,540 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:41,540 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:41,540 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41077,1675954443212, table=testRollbackForSplitTableRegionWithReplica, region=813d8870e2a4f2929f4991fa0f505d12 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=080674692710b4dfd3481fd1d48b1c60, daughterA=45103f245599b3c2b8e9be2a40c7083b, daughterB=47b175695c3d18e949263cbc873d2e48 2023-02-09 14:55:41,540 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:41,541 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:41,541 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:41,541 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:41,541 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:41,541 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 14:55:41,542 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,542 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:55:41,542 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,542 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,542 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,542 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-09 14:55:41,543 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:55:41,543 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:55:41,544 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954571544 2023-02-09 14:55:41,545 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,545 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:55:41,548 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:55:41,548 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=a38bfbdf613fc88c3b7b74edf1f8bd28, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33553,1675954443180, regionLocation=jenkins-hbase4.apache.org,33553,1675954443180, openSeqNum=2 2023-02-09 14:55:41,548 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:55:41,548 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954601548 2023-02-09 14:55:41,548 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954661548 2023-02-09 14:55:41,548 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 14:55:41,548 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28, UNASSIGN}] 2023-02-09 14:55:41,548 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:55:41,550 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28, UNASSIGN 2023-02-09 14:55:41,551 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=a38bfbdf613fc88c3b7b74edf1f8bd28, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:41,551 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675954541551"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954541551"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954541551"}]},"ts":"1675954541551"} 2023-02-09 14:55:41,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure a38bfbdf613fc88c3b7b74edf1f8bd28, server=jenkins-hbase4.apache.org,33553,1675954443180}] 2023-02-09 14:55:41,704 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a38bfbdf613fc88c3b7b74edf1f8bd28, disabling compactions & flushes 2023-02-09 14:55:41,704 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. 2023-02-09 14:55:41,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. 2023-02-09 14:55:41,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. after waiting 0 ms 2023-02-09 14:55:41,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. 2023-02-09 14:55:41,704 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing a38bfbdf613fc88c3b7b74edf1f8bd28 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 14:55:41,714 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf1/ef098f0b243849bc90a19fb43914356c 2023-02-09 14:55:41,727 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf2/f0401a668795474bba66cfc5da136faf 2023-02-09 14:55:41,731 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf1/ef098f0b243849bc90a19fb43914356c as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/ef098f0b243849bc90a19fb43914356c 2023-02-09 14:55:41,733 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/ef098f0b243849bc90a19fb43914356c, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 14:55:41,734 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/.tmp/cf2/f0401a668795474bba66cfc5da136faf as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/f0401a668795474bba66cfc5da136faf 2023-02-09 14:55:41,737 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/f0401a668795474bba66cfc5da136faf, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 14:55:41,738 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for a38bfbdf613fc88c3b7b74edf1f8bd28 in 34ms, sequenceid=82, compaction requested=true 2023-02-09 14:55:41,743 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 14:55:41,743 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:55:41,744 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. 2023-02-09 14:55:41,745 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a38bfbdf613fc88c3b7b74edf1f8bd28: Waiting for close lock at 1675954541704Running coprocessor pre-close hooks at 1675954541704Disabling compacts and flushes for region at 1675954541704Disabling writes for close at 1675954541704Obtaining lock to block concurrent updates at 1675954541704Preparing flush snapshotting stores in a38bfbdf613fc88c3b7b74edf1f8bd28 at 1675954541704Finished memstore snapshotting testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675954541704Flushing stores of testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. at 1675954541705 (+1 ms)Flushing a38bfbdf613fc88c3b7b74edf1f8bd28/cf1: creating writer at 1675954541706 (+1 ms)Flushing a38bfbdf613fc88c3b7b74edf1f8bd28/cf1: appending metadata at 1675954541708 (+2 ms)Flushing a38bfbdf613fc88c3b7b74edf1f8bd28/cf1: closing flushed file at 1675954541708Flushing a38bfbdf613fc88c3b7b74edf1f8bd28/cf2: creating writer at 1675954541719 (+11 ms)Flushing a38bfbdf613fc88c3b7b74edf1f8bd28/cf2: appending metadata at 1675954541720 (+1 ms)Flushing a38bfbdf613fc88c3b7b74edf1f8bd28/cf2: closing flushed file at 1675954541720Flushing a38bfbdf613fc88c3b7b74edf1f8bd28/cf1: reopening flushed file at 1675954541731 (+11 ms)Flushing a38bfbdf613fc88c3b7b74edf1f8bd28/cf2: reopening flushed file at 1675954541734 (+3 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for a38bfbdf613fc88c3b7b74edf1f8bd28 in 34ms, sequenceid=82, compaction requested=true at 1675954541738 (+4 ms)Writing region close event to WAL at 1675954541740 (+2 ms)Running coprocessor post-close hooks at 1675954541743 (+3 ms)Closed at 1675954541744 (+1 ms) 2023-02-09 14:55:41,749 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,749 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=a38bfbdf613fc88c3b7b74edf1f8bd28, regionState=CLOSED 2023-02-09 14:55:41,749 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675954541749"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954541749"}]},"ts":"1675954541749"} 2023-02-09 14:55:41,752 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-09 14:55:41,752 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a38bfbdf613fc88c3b7b74edf1f8bd28, server=jenkins-hbase4.apache.org,33553,1675954443180 in 198 msec 2023-02-09 14:55:41,753 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-09 14:55:41,753 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28, UNASSIGN in 204 msec 2023-02-09 14:55:41,755 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:55:41,755 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,755 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:55:41,755 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:55:41,798 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,798 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:55:41,798 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:55:41,799 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:55:41,799 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:55:41,799 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:55:41,799 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:55:41,799 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:55:41,799 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:55:41,799 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:55:41,801 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-09 14:55:41,801 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 14:55:41,801 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 14:55:41,801 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:55:41,801 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:55:41,801 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:55:41,801 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:55:41,802 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:41,802 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:55:41,802 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:41,802 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:41,802 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:41,802 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:55:41,802 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:41,803 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:41,803 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:41,803 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:55:41,803 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN 2023-02-09 14:55:41,804 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:41,804 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:41,804 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:41,804 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7a3a2a2c1be8b68cf6b9c5576ca1d967, daughterA=915612316d6a09fe13e83a17e2d06051, daughterB=2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:55:41,804 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:41,805 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:41,805 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:41,805 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=94ffc82cbf109e7f71870e19837a0e78, daughterA=c720e43e6c74641ba50283474b45a523, daughterB=8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:55:41,805 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:41,806 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:41,806 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:41,806 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:41,806 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41077,1675954443212, table=testRollbackForSplitTableRegionWithReplica, region=813d8870e2a4f2929f4991fa0f505d12 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=080674692710b4dfd3481fd1d48b1c60, daughterA=45103f245599b3c2b8e9be2a40c7083b, daughterB=47b175695c3d18e949263cbc873d2e48 2023-02-09 14:55:41,807 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:41,807 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:41,807 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:41,807 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:41,807 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:41,807 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 14:55:41,808 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,808 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28, UNASSIGN 2023-02-09 14:55:41,808 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a38bfbdf613fc88c3b7b74edf1f8bd28, server=jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:41,808 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:55:41,808 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,808 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,808 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,809 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:55:41,809 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a38bfbdf613fc88c3b7b74edf1f8bd28, server=jenkins-hbase4.apache.org,33553,1675954443180 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:55:41,809 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-09 14:55:41,829 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:55:41,830 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:55:41,831 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954571831 2023-02-09 14:55:41,832 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:41,832 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:55:41,835 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:55:41,835 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=a38bfbdf613fc88c3b7b74edf1f8bd28, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33553,1675954443180, regionLocation=jenkins-hbase4.apache.org,33553,1675954443180, openSeqNum=2 2023-02-09 14:55:41,835 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:55:41,835 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954601835 2023-02-09 14:55:41,835 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954661835 2023-02-09 14:55:41,835 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 14:55:41,837 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:55:41,844 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=a38bfbdf613fc88c3b7b74edf1f8bd28, threads=14 2023-02-09 14:55:41,845 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/333988a5c29b47449390ab71b1a3ef2f for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,845 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/bafbe116570d4368879cf0bbc41638d3 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,845 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/d30ffcea033d4133941ec24b991abfbe for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,845 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e59ca37300bd48b6a4ee7ac39554cbaf for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,845 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e8315fec79ba4d5ab09ccd57719ed2a6 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,845 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/f0401a668795474bba66cfc5da136faf for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,845 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/fdc7589cbab3484eb9c2b4473a70e219 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,845 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/09103225a5724a4d8c9313c7f68a607e for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,845 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/5b999bec37ed41eca0c9366717969996 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,845 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/8c0ffa708ab14279b63e065ccfd3ce98 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,846 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/a551eb3bbc834e1f80951082e085d1fd for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,846 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/d2e156e7ebf64b6ba510ea828773eca1 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,846 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,846 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/ef098f0b243849bc90a19fb43914356c for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,920 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/5b999bec37ed41eca0c9366717969996 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,924 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/f0401a668795474bba66cfc5da136faf for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,937 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e8315fec79ba4d5ab09ccd57719ed2a6 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,938 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/d2e156e7ebf64b6ba510ea828773eca1 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,941 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/bafbe116570d4368879cf0bbc41638d3 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,943 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/09103225a5724a4d8c9313c7f68a607e for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,943 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,944 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e59ca37300bd48b6a4ee7ac39554cbaf for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,947 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/a551eb3bbc834e1f80951082e085d1fd for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,948 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/fdc7589cbab3484eb9c2b4473a70e219 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,951 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/8c0ffa708ab14279b63e065ccfd3ce98 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,953 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/ef098f0b243849bc90a19fb43914356c for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:41,958 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/333988a5c29b47449390ab71b1a3ef2f for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,169 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 14:55:42,334 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/d30ffcea033d4133941ec24b991abfbe for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,334 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region a38bfbdf613fc88c3b7b74edf1f8bd28 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 14:55:42,361 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:55:42,361 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:42,361 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:55:42,361 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:55:42,586 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:42,586 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:55:42,586 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:55:42,586 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:55:42,586 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:55:42,586 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:55:42,586 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:55:42,586 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:55:42,586 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:55:42,587 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:55:42,588 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-09 14:55:42,588 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 14:55:42,588 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 14:55:42,588 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:55:42,589 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:55:42,589 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:55:42,589 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:55:42,589 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:42,589 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:55:42,589 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:42,589 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:42,590 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:42,590 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:55:42,590 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:42,590 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:42,590 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:42,590 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:55:42,590 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN 2023-02-09 14:55:42,590 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:42,591 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:42,591 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:42,591 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7a3a2a2c1be8b68cf6b9c5576ca1d967, daughterA=915612316d6a09fe13e83a17e2d06051, daughterB=2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:55:42,591 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:42,591 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:42,591 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:42,591 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=94ffc82cbf109e7f71870e19837a0e78, daughterA=c720e43e6c74641ba50283474b45a523, daughterB=8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:55:42,592 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:42,592 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:42,592 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:42,592 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:42,592 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41077,1675954443212, table=testRollbackForSplitTableRegionWithReplica, region=813d8870e2a4f2929f4991fa0f505d12 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=080674692710b4dfd3481fd1d48b1c60, daughterA=45103f245599b3c2b8e9be2a40c7083b, daughterB=47b175695c3d18e949263cbc873d2e48 2023-02-09 14:55:42,592 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:42,593 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:42,593 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:42,593 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:42,593 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:42,593 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 14:55:42,593 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:42,593 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28, UNASSIGN 2023-02-09 14:55:42,594 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a38bfbdf613fc88c3b7b74edf1f8bd28, server=jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:42,594 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:55:42,594 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:42,594 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:42,594 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:42,594 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:55:42,594 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a38bfbdf613fc88c3b7b74edf1f8bd28, server=jenkins-hbase4.apache.org,33553,1675954443180 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:55:42,594 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-09 14:55:42,595 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:55:42,595 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:55:42,595 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954572595 2023-02-09 14:55:42,597 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:42,597 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:55:42,599 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:55:42,599 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=a38bfbdf613fc88c3b7b74edf1f8bd28, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33553,1675954443180, regionLocation=jenkins-hbase4.apache.org,33553,1675954443180, openSeqNum=2 2023-02-09 14:55:42,599 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:55:42,599 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954602599 2023-02-09 14:55:42,599 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954662599 2023-02-09 14:55:42,599 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 14:55:42,606 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=a38bfbdf613fc88c3b7b74edf1f8bd28, threads=14 2023-02-09 14:55:42,607 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/333988a5c29b47449390ab71b1a3ef2f for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,607 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/bafbe116570d4368879cf0bbc41638d3 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,607 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/d30ffcea033d4133941ec24b991abfbe for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,607 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e59ca37300bd48b6a4ee7ac39554cbaf for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,607 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e8315fec79ba4d5ab09ccd57719ed2a6 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,607 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/f0401a668795474bba66cfc5da136faf for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,608 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/fdc7589cbab3484eb9c2b4473a70e219 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,609 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/09103225a5724a4d8c9313c7f68a607e for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,609 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/5b999bec37ed41eca0c9366717969996 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,609 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/8c0ffa708ab14279b63e065ccfd3ce98 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,609 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/a551eb3bbc834e1f80951082e085d1fd for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,609 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/d2e156e7ebf64b6ba510ea828773eca1 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,610 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,610 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/ef098f0b243849bc90a19fb43914356c for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,700 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/333988a5c29b47449390ab71b1a3ef2f for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,706 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e8315fec79ba4d5ab09ccd57719ed2a6 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,709 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/d30ffcea033d4133941ec24b991abfbe for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,715 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/fdc7589cbab3484eb9c2b4473a70e219 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,717 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/ef098f0b243849bc90a19fb43914356c for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,720 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/d2e156e7ebf64b6ba510ea828773eca1 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,721 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e59ca37300bd48b6a4ee7ac39554cbaf for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,725 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,725 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/09103225a5724a4d8c9313c7f68a607e for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,726 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/bafbe116570d4368879cf0bbc41638d3 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,727 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/8c0ffa708ab14279b63e065ccfd3ce98 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,727 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/f0401a668795474bba66cfc5da136faf for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,728 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/a551eb3bbc834e1f80951082e085d1fd for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:42,865 ERROR [BP-2069929948-172.31.14.131-1675954438937 heartbeating to localhost/127.0.0.1:38049] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2069929948-172.31.14.131-1675954438937 (Datanode Uuid 30e687f1-c071-43d7-89db-78c265b7458a) service to localhost/127.0.0.1:38049 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-09 14:55:43,109 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/5b999bec37ed41eca0c9366717969996 for region: a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:43,110 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region a38bfbdf613fc88c3b7b74edf1f8bd28 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 14:55:43,139 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:55:43,144 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 14:55:43,145 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 14:55:43,145 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:55:43,145 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:43,145 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:55:43,145 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:55:43,350 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:43,350 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:55:43,350 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:55:43,350 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:55:43,350 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:55:43,350 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:55:43,350 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:55:43,350 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:55:43,351 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:55:43,351 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:55:43,352 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-09 14:55:43,353 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 14:55:43,353 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 14:55:43,353 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:55:43,353 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:55:43,353 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:55:43,353 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:55:43,354 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:43,354 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:55:43,354 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:43,354 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:43,355 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:43,355 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:55:43,355 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:43,355 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:43,355 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:43,355 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:55:43,356 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN 2023-02-09 14:55:43,356 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:43,356 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:43,356 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:43,356 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7a3a2a2c1be8b68cf6b9c5576ca1d967, daughterA=915612316d6a09fe13e83a17e2d06051, daughterB=2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:55:43,357 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:43,357 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:43,357 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:43,357 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=94ffc82cbf109e7f71870e19837a0e78, daughterA=c720e43e6c74641ba50283474b45a523, daughterB=8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:55:43,357 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:43,357 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:43,358 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:43,358 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:43,358 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41077,1675954443212, table=testRollbackForSplitTableRegionWithReplica, region=813d8870e2a4f2929f4991fa0f505d12 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=080674692710b4dfd3481fd1d48b1c60, daughterA=45103f245599b3c2b8e9be2a40c7083b, daughterB=47b175695c3d18e949263cbc873d2e48 2023-02-09 14:55:43,358 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:43,359 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:43,359 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:43,359 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:43,359 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:43,359 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 14:55:43,360 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:43,360 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28, UNASSIGN 2023-02-09 14:55:43,360 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a38bfbdf613fc88c3b7b74edf1f8bd28, server=jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:43,360 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:55:43,360 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:43,360 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:43,360 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:43,360 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:55:43,360 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a38bfbdf613fc88c3b7b74edf1f8bd28, server=jenkins-hbase4.apache.org,33553,1675954443180 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:55:43,361 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-09 14:55:43,361 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:55:43,362 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:55:43,362 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954573362 2023-02-09 14:55:43,363 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:55:43,364 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:43,370 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:55:43,370 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=a38bfbdf613fc88c3b7b74edf1f8bd28, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33553,1675954443180, regionLocation=jenkins-hbase4.apache.org,33553,1675954443180, openSeqNum=2 2023-02-09 14:55:43,370 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:55:43,370 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954603370 2023-02-09 14:55:43,370 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954663370 2023-02-09 14:55:43,370 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 7 msec 2023-02-09 14:55:43,373 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:55:43,374 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:55:43,374 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:43,374 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:55:43,375 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:55:43,620 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:43,621 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:55:43,621 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:55:43,621 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:55:43,621 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:55:43,621 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:55:43,621 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:55:43,621 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:55:43,621 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:55:43,622 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:55:43,623 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-09 14:55:43,623 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 14:55:43,623 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 14:55:43,623 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:55:43,623 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:55:43,623 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:55:43,623 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:55:43,624 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:43,624 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:55:43,624 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:43,624 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:43,624 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:43,624 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:55:43,624 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:43,624 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:43,625 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:43,625 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:55:43,625 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN 2023-02-09 14:55:43,625 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:43,625 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:43,625 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:43,625 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7a3a2a2c1be8b68cf6b9c5576ca1d967, daughterA=915612316d6a09fe13e83a17e2d06051, daughterB=2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:55:43,625 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:43,625 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:43,626 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:43,626 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=94ffc82cbf109e7f71870e19837a0e78, daughterA=c720e43e6c74641ba50283474b45a523, daughterB=8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:55:43,626 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:43,626 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:43,626 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:43,626 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:43,626 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41077,1675954443212, table=testRollbackForSplitTableRegionWithReplica, region=813d8870e2a4f2929f4991fa0f505d12 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=080674692710b4dfd3481fd1d48b1c60, daughterA=45103f245599b3c2b8e9be2a40c7083b, daughterB=47b175695c3d18e949263cbc873d2e48 2023-02-09 14:55:43,627 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:43,627 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:43,627 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:43,627 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:43,627 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:43,627 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 14:55:43,627 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:43,627 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28, UNASSIGN 2023-02-09 14:55:43,628 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a38bfbdf613fc88c3b7b74edf1f8bd28, server=jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:43,628 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:55:43,628 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:43,628 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:43,628 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:43,628 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:55:43,628 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a38bfbdf613fc88c3b7b74edf1f8bd28, server=jenkins-hbase4.apache.org,33553,1675954443180 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:55:43,628 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 14:55:43,629 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:55:43,629 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:55:43,629 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954573629 2023-02-09 14:55:43,631 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:55:43,631 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:43,633 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:55:43,633 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=a38bfbdf613fc88c3b7b74edf1f8bd28, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33553,1675954443180, regionLocation=jenkins-hbase4.apache.org,33553,1675954443180, openSeqNum=2 2023-02-09 14:55:43,633 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:55:43,633 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954603633 2023-02-09 14:55:43,633 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954663633 2023-02-09 14:55:43,633 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-09 14:55:43,633 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:55:43,635 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675954543635"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1675954543635"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1675954543635"}]},"ts":"1675954543635"} 2023-02-09 14:55:43,635 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675954543635"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954543635"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954543635"}]},"ts":"1675954543635"} 2023-02-09 14:55:43,635 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675954543635"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954543635"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954543635"}]},"ts":"1675954543635"} 2023-02-09 14:55:43,636 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:55:43,637 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:43,637 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:55:43,637 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:55:43,883 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:43,883 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:55:43,884 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:55:43,884 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:55:43,884 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:55:43,884 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:55:43,884 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:55:43,884 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:55:43,884 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:55:43,885 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:55:43,886 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-09 14:55:43,886 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 14:55:43,886 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 14:55:43,886 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:55:43,887 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:55:43,887 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:55:43,887 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:55:43,887 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:43,888 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:55:43,888 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:43,888 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:43,888 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:43,888 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:55:43,889 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:43,889 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:43,889 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:43,889 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:55:43,890 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN 2023-02-09 14:55:43,890 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:43,890 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:43,890 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:43,890 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7a3a2a2c1be8b68cf6b9c5576ca1d967, daughterA=915612316d6a09fe13e83a17e2d06051, daughterB=2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:55:43,890 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:43,890 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:43,890 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:43,891 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=94ffc82cbf109e7f71870e19837a0e78, daughterA=c720e43e6c74641ba50283474b45a523, daughterB=8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:55:43,891 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:43,891 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:43,891 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:43,891 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:43,892 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41077,1675954443212, table=testRollbackForSplitTableRegionWithReplica, region=813d8870e2a4f2929f4991fa0f505d12 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=080674692710b4dfd3481fd1d48b1c60, daughterA=45103f245599b3c2b8e9be2a40c7083b, daughterB=47b175695c3d18e949263cbc873d2e48 2023-02-09 14:55:43,892 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:43,892 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:43,892 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:43,892 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:43,892 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:43,892 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 14:55:43,893 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:43,893 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28, UNASSIGN 2023-02-09 14:55:43,893 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a38bfbdf613fc88c3b7b74edf1f8bd28, server=jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:43,893 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:55:43,893 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:43,893 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:43,893 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:43,893 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:55:43,893 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a38bfbdf613fc88c3b7b74edf1f8bd28, server=jenkins-hbase4.apache.org,33553,1675954443180 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:55:43,894 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-09 14:55:43,894 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:55:43,895 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:55:43,895 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954573895 2023-02-09 14:55:43,896 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:55:43,897 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:43,900 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:55:43,900 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=a38bfbdf613fc88c3b7b74edf1f8bd28, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33553,1675954443180, regionLocation=jenkins-hbase4.apache.org,33553,1675954443180, openSeqNum=2 2023-02-09 14:55:43,900 INFO [Listener at localhost/40823] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,33553,1675954443180, table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:43,900 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=8c791f527032c6c0e52c041969bac8e3, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-09 14:55:43,901 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=6c7521c64dfe564c431c1e4733b56ec3, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-09 14:55:43,901 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:55:43,901 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954603901 2023-02-09 14:55:43,901 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954663901 2023-02-09 14:55:43,901 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 14:55:43,901 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675954543901"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1675954543901"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1675954543901"}]},"ts":"1675954543901"} 2023-02-09 14:55:43,901 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675954543901"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954543901"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954543901"}]},"ts":"1675954543901"} 2023-02-09 14:55:43,901 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675954543901"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954543901"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954543901"}]},"ts":"1675954543901"} 2023-02-09 14:55:43,903 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:55:43,905 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:55:43,905 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:43,905 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:55:43,905 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:55:44,151 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,151 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:55:44,151 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:55:44,151 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:55:44,152 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:55:44,152 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:55:44,152 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:55:44,152 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:55:44,152 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:55:44,152 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:55:44,154 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-02-09 14:55:44,154 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 14:55:44,154 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 14:55:44,154 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:55:44,155 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:55:44,155 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:55:44,155 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:55:44,155 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:44,156 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:55:44,156 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:44,156 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:44,156 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:44,156 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:55:44,156 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:44,156 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:44,157 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:44,157 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:55:44,157 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN 2023-02-09 14:55:44,157 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:44,157 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:44,157 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:44,157 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7a3a2a2c1be8b68cf6b9c5576ca1d967, daughterA=915612316d6a09fe13e83a17e2d06051, daughterB=2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:55:44,157 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:44,157 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:44,158 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:44,158 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=94ffc82cbf109e7f71870e19837a0e78, daughterA=c720e43e6c74641ba50283474b45a523, daughterB=8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:55:44,158 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:44,158 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:44,158 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:44,158 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:44,159 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41077,1675954443212, table=testRollbackForSplitTableRegionWithReplica, region=813d8870e2a4f2929f4991fa0f505d12 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=080674692710b4dfd3481fd1d48b1c60, daughterA=45103f245599b3c2b8e9be2a40c7083b, daughterB=47b175695c3d18e949263cbc873d2e48 2023-02-09 14:55:44,159 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:44,159 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:44,159 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:44,159 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:44,159 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:44,159 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 14:55:44,159 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,159 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28, UNASSIGN 2023-02-09 14:55:44,160 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a38bfbdf613fc88c3b7b74edf1f8bd28, server=jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:44,160 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:55:44,160 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,160 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,160 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,160 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:55:44,160 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a38bfbdf613fc88c3b7b74edf1f8bd28, server=jenkins-hbase4.apache.org,33553,1675954443180 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:55:44,160 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-09 14:55:44,161 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:55:44,161 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:55:44,161 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954574161 2023-02-09 14:55:44,161 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:55:44,163 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,165 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:55:44,165 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=a38bfbdf613fc88c3b7b74edf1f8bd28, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33553,1675954443180, regionLocation=jenkins-hbase4.apache.org,33553,1675954443180, openSeqNum=2 2023-02-09 14:55:44,165 INFO [Listener at localhost/40823] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,33553,1675954443180, table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:44,165 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=8c791f527032c6c0e52c041969bac8e3, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-09 14:55:44,165 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=6c7521c64dfe564c431c1e4733b56ec3, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-09 14:55:44,166 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:55:44,166 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954604166 2023-02-09 14:55:44,166 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954664166 2023-02-09 14:55:44,166 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 14:55:44,166 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:55:44,168 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8c791f527032c6c0e52c041969bac8e3, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6c7521c64dfe564c431c1e4733b56ec3, ASSIGN}] 2023-02-09 14:55:44,168 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:55:44,168 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,168 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:55:44,168 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:55:44,416 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,416 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:55:44,416 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:55:44,416 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:55:44,416 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:55:44,416 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:55:44,416 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:55:44,417 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:55:44,417 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:55:44,417 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:55:44,418 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-02-09 14:55:44,418 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 14:55:44,418 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 14:55:44,419 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:55:44,419 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:55:44,419 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:55:44,419 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:55:44,419 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:44,419 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:55:44,420 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:44,420 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:44,420 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:44,420 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:55:44,420 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:44,420 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:44,420 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:44,420 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:55:44,421 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN 2023-02-09 14:55:44,421 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:44,421 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:44,421 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:44,421 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7a3a2a2c1be8b68cf6b9c5576ca1d967, daughterA=915612316d6a09fe13e83a17e2d06051, daughterB=2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:55:44,421 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:44,421 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:44,421 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:44,421 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=94ffc82cbf109e7f71870e19837a0e78, daughterA=c720e43e6c74641ba50283474b45a523, daughterB=8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:55:44,421 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:44,422 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:44,422 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:44,422 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:44,422 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41077,1675954443212, table=testRollbackForSplitTableRegionWithReplica, region=813d8870e2a4f2929f4991fa0f505d12 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=080674692710b4dfd3481fd1d48b1c60, daughterA=45103f245599b3c2b8e9be2a40c7083b, daughterB=47b175695c3d18e949263cbc873d2e48 2023-02-09 14:55:44,422 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:44,422 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:44,422 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:44,422 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:44,423 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:44,423 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 14:55:44,423 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,423 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28, UNASSIGN 2023-02-09 14:55:44,423 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a38bfbdf613fc88c3b7b74edf1f8bd28, server=jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:44,423 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:55:44,423 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,423 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,423 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,423 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:55:44,423 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a38bfbdf613fc88c3b7b74edf1f8bd28, server=jenkins-hbase4.apache.org,33553,1675954443180 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:55:44,424 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-09 14:55:44,424 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:55:44,424 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:55:44,425 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954574425 2023-02-09 14:55:44,425 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:55:44,426 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,428 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:55:44,429 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=a38bfbdf613fc88c3b7b74edf1f8bd28, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33553,1675954443180, regionLocation=jenkins-hbase4.apache.org,33553,1675954443180, openSeqNum=2 2023-02-09 14:55:44,429 INFO [Listener at localhost/40823] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,33553,1675954443180, table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:44,429 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=8c791f527032c6c0e52c041969bac8e3, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-09 14:55:44,429 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=6c7521c64dfe564c431c1e4733b56ec3, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-09 14:55:44,429 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:55:44,429 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954604429 2023-02-09 14:55:44,430 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954664430 2023-02-09 14:55:44,430 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-09 14:55:44,430 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8c791f527032c6c0e52c041969bac8e3, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6c7521c64dfe564c431c1e4733b56ec3, ASSIGN}] 2023-02-09 14:55:44,430 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:55:44,432 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6c7521c64dfe564c431c1e4733b56ec3, ASSIGN 2023-02-09 14:55:44,432 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6c7521c64dfe564c431c1e4733b56ec3, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,33553,1675954443180; forceNewPlan=false, retain=false 2023-02-09 14:55:44,433 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8c791f527032c6c0e52c041969bac8e3, ASSIGN 2023-02-09 14:55:44,433 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8c791f527032c6c0e52c041969bac8e3, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,33553,1675954443180; forceNewPlan=false, retain=false 2023-02-09 14:55:44,583 INFO [jenkins-hbase4:43829] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 14:55:44,584 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=8c791f527032c6c0e52c041969bac8e3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:44,584 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675954544584"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954544584"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954544584"}]},"ts":"1675954544584"} 2023-02-09 14:55:44,586 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=115, state=RUNNABLE; OpenRegionProcedure 8c791f527032c6c0e52c041969bac8e3, server=jenkins-hbase4.apache.org,33553,1675954443180}] 2023-02-09 14:55:44,588 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=6c7521c64dfe564c431c1e4733b56ec3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:44,588 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675954544587"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954544587"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954544587"}]},"ts":"1675954544587"} 2023-02-09 14:55:44,589 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=116, state=RUNNABLE; OpenRegionProcedure 6c7521c64dfe564c431c1e4733b56ec3, server=jenkins-hbase4.apache.org,33553,1675954443180}] 2023-02-09 14:55:44,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3. 2023-02-09 14:55:44,739 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8c791f527032c6c0e52c041969bac8e3, NAME => 'testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.', STARTKEY => '', ENDKEY => '41'} 2023-02-09 14:55:44,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:55:44,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 8c791f527032c6c0e52c041969bac8e3 2023-02-09 14:55:44,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:44,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 8c791f527032c6c0e52c041969bac8e3 2023-02-09 14:55:44,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 8c791f527032c6c0e52c041969bac8e3 2023-02-09 14:55:44,741 INFO [StoreOpener-8c791f527032c6c0e52c041969bac8e3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8c791f527032c6c0e52c041969bac8e3 2023-02-09 14:55:44,742 DEBUG [StoreOpener-8c791f527032c6c0e52c041969bac8e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1 2023-02-09 14:55:44,742 DEBUG [StoreOpener-8c791f527032c6c0e52c041969bac8e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1 2023-02-09 14:55:44,742 INFO [StoreOpener-8c791f527032c6c0e52c041969bac8e3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8c791f527032c6c0e52c041969bac8e3 columnFamilyName cf1 2023-02-09 14:55:44,753 DEBUG [StoreOpener-8c791f527032c6c0e52c041969bac8e3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/09103225a5724a4d8c9313c7f68a607e.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/09103225a5724a4d8c9313c7f68a607e-bottom 2023-02-09 14:55:44,756 DEBUG [StoreOpener-8c791f527032c6c0e52c041969bac8e3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/5b999bec37ed41eca0c9366717969996.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/5b999bec37ed41eca0c9366717969996-bottom 2023-02-09 14:55:44,760 DEBUG [StoreOpener-8c791f527032c6c0e52c041969bac8e3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/8c0ffa708ab14279b63e065ccfd3ce98.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/8c0ffa708ab14279b63e065ccfd3ce98-bottom 2023-02-09 14:55:44,763 DEBUG [StoreOpener-8c791f527032c6c0e52c041969bac8e3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/a551eb3bbc834e1f80951082e085d1fd.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/a551eb3bbc834e1f80951082e085d1fd-bottom 2023-02-09 14:55:44,766 DEBUG [StoreOpener-8c791f527032c6c0e52c041969bac8e3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/d2e156e7ebf64b6ba510ea828773eca1.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/d2e156e7ebf64b6ba510ea828773eca1-bottom 2023-02-09 14:55:44,769 DEBUG [StoreOpener-8c791f527032c6c0e52c041969bac8e3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5-bottom 2023-02-09 14:55:44,772 DEBUG [StoreOpener-8c791f527032c6c0e52c041969bac8e3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/ef098f0b243849bc90a19fb43914356c.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/ef098f0b243849bc90a19fb43914356c-bottom 2023-02-09 14:55:44,773 INFO [StoreOpener-8c791f527032c6c0e52c041969bac8e3-1] regionserver.HStore(310): Store=8c791f527032c6c0e52c041969bac8e3/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:44,773 INFO [StoreOpener-8c791f527032c6c0e52c041969bac8e3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8c791f527032c6c0e52c041969bac8e3 2023-02-09 14:55:44,773 DEBUG [StoreOpener-8c791f527032c6c0e52c041969bac8e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2 2023-02-09 14:55:44,773 DEBUG [StoreOpener-8c791f527032c6c0e52c041969bac8e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2 2023-02-09 14:55:44,774 INFO [StoreOpener-8c791f527032c6c0e52c041969bac8e3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8c791f527032c6c0e52c041969bac8e3 columnFamilyName cf2 2023-02-09 14:55:44,784 DEBUG [StoreOpener-8c791f527032c6c0e52c041969bac8e3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/333988a5c29b47449390ab71b1a3ef2f.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/333988a5c29b47449390ab71b1a3ef2f-bottom 2023-02-09 14:55:44,787 DEBUG [StoreOpener-8c791f527032c6c0e52c041969bac8e3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/bafbe116570d4368879cf0bbc41638d3.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/bafbe116570d4368879cf0bbc41638d3-bottom 2023-02-09 14:55:44,790 DEBUG [StoreOpener-8c791f527032c6c0e52c041969bac8e3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/d30ffcea033d4133941ec24b991abfbe.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/d30ffcea033d4133941ec24b991abfbe-bottom 2023-02-09 14:55:44,793 DEBUG [StoreOpener-8c791f527032c6c0e52c041969bac8e3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/e59ca37300bd48b6a4ee7ac39554cbaf.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e59ca37300bd48b6a4ee7ac39554cbaf-bottom 2023-02-09 14:55:44,797 DEBUG [StoreOpener-8c791f527032c6c0e52c041969bac8e3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/e8315fec79ba4d5ab09ccd57719ed2a6.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e8315fec79ba4d5ab09ccd57719ed2a6-bottom 2023-02-09 14:55:44,800 DEBUG [StoreOpener-8c791f527032c6c0e52c041969bac8e3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/f0401a668795474bba66cfc5da136faf.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/f0401a668795474bba66cfc5da136faf-bottom 2023-02-09 14:55:44,803 DEBUG [StoreOpener-8c791f527032c6c0e52c041969bac8e3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/fdc7589cbab3484eb9c2b4473a70e219.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/fdc7589cbab3484eb9c2b4473a70e219-bottom 2023-02-09 14:55:44,803 INFO [StoreOpener-8c791f527032c6c0e52c041969bac8e3-1] regionserver.HStore(310): Store=8c791f527032c6c0e52c041969bac8e3/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:44,804 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3 2023-02-09 14:55:44,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3 2023-02-09 14:55:44,807 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:55:44,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 8c791f527032c6c0e52c041969bac8e3 2023-02-09 14:55:44,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 8c791f527032c6c0e52c041969bac8e3; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68243937, jitterRate=0.01691390573978424}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:55:44,809 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 8c791f527032c6c0e52c041969bac8e3: Running coprocessor pre-open hook at 1675954544740Writing region info on filesystem at 1675954544740Initializing all the Stores at 1675954544741 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954544741Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954544741Cleaning up temporary data from old regions at 1675954544806 (+65 ms)Cleaning up detritus from prior splits at 1675954544806Running coprocessor post-open hooks at 1675954544809 (+3 ms)Region opened successfully at 1675954544809 2023-02-09 14:55:44,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3., pid=117, masterSystemTime=1675954544737 2023-02-09 14:55:44,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:55:44,813 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 14:55:44,813 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 14:55:44,814 DEBUG [RS:1;jenkins-hbase4:33553-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 14:55:44,814 INFO [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3. 2023-02-09 14:55:44,814 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.HStore(1912): 8c791f527032c6c0e52c041969bac8e3/cf1 is initiating minor compaction (all files) 2023-02-09 14:55:44,814 INFO [RS:1;jenkins-hbase4:33553-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3. 2023-02-09 14:55:44,814 DEBUG [RS:1;jenkins-hbase4:33553-longCompactions-0] regionserver.HStore(1912): 8c791f527032c6c0e52c041969bac8e3/cf2 is initiating minor compaction (all files) 2023-02-09 14:55:44,814 INFO [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 8c791f527032c6c0e52c041969bac8e3/cf1 in testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3. 2023-02-09 14:55:44,814 INFO [RS:1;jenkins-hbase4:33553-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 8c791f527032c6c0e52c041969bac8e3/cf2 in testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3. 2023-02-09 14:55:44,814 INFO [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/a551eb3bbc834e1f80951082e085d1fd.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/a551eb3bbc834e1f80951082e085d1fd-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/d2e156e7ebf64b6ba510ea828773eca1.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/d2e156e7ebf64b6ba510ea828773eca1-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/5b999bec37ed41eca0c9366717969996.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/5b999bec37ed41eca0c9366717969996-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/09103225a5724a4d8c9313c7f68a607e.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/09103225a5724a4d8c9313c7f68a607e-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/8c0ffa708ab14279b63e065ccfd3ce98.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/8c0ffa708ab14279b63e065ccfd3ce98-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/ef098f0b243849bc90a19fb43914356c.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/ef098f0b243849bc90a19fb43914356c-bottom] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/.tmp, totalSize=34.9 K 2023-02-09 14:55:44,814 INFO [RS:1;jenkins-hbase4:33553-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/e59ca37300bd48b6a4ee7ac39554cbaf.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e59ca37300bd48b6a4ee7ac39554cbaf-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/d30ffcea033d4133941ec24b991abfbe.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/d30ffcea033d4133941ec24b991abfbe-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/333988a5c29b47449390ab71b1a3ef2f.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/333988a5c29b47449390ab71b1a3ef2f-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/fdc7589cbab3484eb9c2b4473a70e219.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/fdc7589cbab3484eb9c2b4473a70e219-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/bafbe116570d4368879cf0bbc41638d3.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/bafbe116570d4368879cf0bbc41638d3-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/e8315fec79ba4d5ab09ccd57719ed2a6.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e8315fec79ba4d5ab09ccd57719ed2a6-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/f0401a668795474bba66cfc5da136faf.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/f0401a668795474bba66cfc5da136faf-bottom] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/.tmp, totalSize=34.9 K 2023-02-09 14:55:44,815 DEBUG [RS:1;jenkins-hbase4:33553-longCompactions-0] compactions.Compactor(207): Compacting e59ca37300bd48b6a4ee7ac39554cbaf.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675954539464 2023-02-09 14:55:44,815 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] compactions.Compactor(207): Compacting e2b4de5742234f5a9ac8c98a9dfa7ae5.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675954539464 2023-02-09 14:55:44,815 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3. 2023-02-09 14:55:44,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3. 2023-02-09 14:55:44,815 DEBUG [RS:1;jenkins-hbase4:33553-longCompactions-0] compactions.Compactor(207): Compacting d30ffcea033d4133941ec24b991abfbe.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675954539631 2023-02-09 14:55:44,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3. 2023-02-09 14:55:44,815 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] compactions.Compactor(207): Compacting a551eb3bbc834e1f80951082e085d1fd.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675954539631 2023-02-09 14:55:44,815 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6c7521c64dfe564c431c1e4733b56ec3, NAME => 'testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.', STARTKEY => '41', ENDKEY => ''} 2023-02-09 14:55:44,816 DEBUG [RS:1;jenkins-hbase4:33553-longCompactions-0] compactions.Compactor(207): Compacting 333988a5c29b47449390ab71b1a3ef2f.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675954539804 2023-02-09 14:55:44,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:55:44,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:44,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,816 DEBUG [RS:1;jenkins-hbase4:33553-longCompactions-0] compactions.Compactor(207): Compacting fdc7589cbab3484eb9c2b4473a70e219.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675954539988 2023-02-09 14:55:44,815 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=8c791f527032c6c0e52c041969bac8e3, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:44,816 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] compactions.Compactor(207): Compacting d2e156e7ebf64b6ba510ea828773eca1.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675954539804 2023-02-09 14:55:44,816 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675954544815"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954544815"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954544815"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954544815"}]},"ts":"1675954544815"} 2023-02-09 14:55:44,817 DEBUG [RS:1;jenkins-hbase4:33553-longCompactions-0] compactions.Compactor(207): Compacting bafbe116570d4368879cf0bbc41638d3.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675954540159 2023-02-09 14:55:44,817 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] compactions.Compactor(207): Compacting 5b999bec37ed41eca0c9366717969996.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675954539988 2023-02-09 14:55:44,817 DEBUG [RS:1;jenkins-hbase4:33553-longCompactions-0] compactions.Compactor(207): Compacting e8315fec79ba4d5ab09ccd57719ed2a6.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675954540330 2023-02-09 14:55:44,817 INFO [StoreOpener-6c7521c64dfe564c431c1e4733b56ec3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,817 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] compactions.Compactor(207): Compacting 09103225a5724a4d8c9313c7f68a607e.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675954540159 2023-02-09 14:55:44,817 DEBUG [RS:1;jenkins-hbase4:33553-longCompactions-0] compactions.Compactor(207): Compacting f0401a668795474bba66cfc5da136faf.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675954540496 2023-02-09 14:55:44,818 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] compactions.Compactor(207): Compacting 8c0ffa708ab14279b63e065ccfd3ce98.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675954540330 2023-02-09 14:55:44,819 DEBUG [StoreOpener-6c7521c64dfe564c431c1e4733b56ec3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1 2023-02-09 14:55:44,819 DEBUG [StoreOpener-6c7521c64dfe564c431c1e4733b56ec3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1 2023-02-09 14:55:44,819 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=115 2023-02-09 14:55:44,819 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 8c791f527032c6c0e52c041969bac8e3, server=jenkins-hbase4.apache.org,33553,1675954443180 in 232 msec 2023-02-09 14:55:44,820 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] compactions.Compactor(207): Compacting ef098f0b243849bc90a19fb43914356c.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675954540496 2023-02-09 14:55:44,820 INFO [StoreOpener-6c7521c64dfe564c431c1e4733b56ec3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6c7521c64dfe564c431c1e4733b56ec3 columnFamilyName cf1 2023-02-09 14:55:44,821 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8c791f527032c6c0e52c041969bac8e3, ASSIGN in 389 msec 2023-02-09 14:55:44,830 INFO [RS:1;jenkins-hbase4:33553-longCompactions-0] throttle.PressureAwareThroughputController(145): 8c791f527032c6c0e52c041969bac8e3#cf2#compaction#108 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:55:44,832 INFO [RS:1;jenkins-hbase4:33553-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8c791f527032c6c0e52c041969bac8e3#cf1#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:55:44,833 DEBUG [StoreOpener-6c7521c64dfe564c431c1e4733b56ec3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/09103225a5724a4d8c9313c7f68a607e.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/09103225a5724a4d8c9313c7f68a607e-top 2023-02-09 14:55:44,838 DEBUG [StoreOpener-6c7521c64dfe564c431c1e4733b56ec3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/5b999bec37ed41eca0c9366717969996.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/5b999bec37ed41eca0c9366717969996-top 2023-02-09 14:55:44,841 DEBUG [StoreOpener-6c7521c64dfe564c431c1e4733b56ec3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/8c0ffa708ab14279b63e065ccfd3ce98.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/8c0ffa708ab14279b63e065ccfd3ce98-top 2023-02-09 14:55:44,850 DEBUG [StoreOpener-6c7521c64dfe564c431c1e4733b56ec3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/a551eb3bbc834e1f80951082e085d1fd.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/a551eb3bbc834e1f80951082e085d1fd-top 2023-02-09 14:55:44,853 DEBUG [StoreOpener-6c7521c64dfe564c431c1e4733b56ec3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/d2e156e7ebf64b6ba510ea828773eca1.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/d2e156e7ebf64b6ba510ea828773eca1-top 2023-02-09 14:55:44,854 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/.tmp/cf1/e381142197cc48b8a18476236bcce5d5 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/e381142197cc48b8a18476236bcce5d5 2023-02-09 14:55:44,856 DEBUG [StoreOpener-6c7521c64dfe564c431c1e4733b56ec3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5-top 2023-02-09 14:55:44,858 INFO [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 8c791f527032c6c0e52c041969bac8e3/cf1 of 8c791f527032c6c0e52c041969bac8e3 into e381142197cc48b8a18476236bcce5d5(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:55:44,858 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 8c791f527032c6c0e52c041969bac8e3: 2023-02-09 14:55:44,858 INFO [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3., storeName=8c791f527032c6c0e52c041969bac8e3/cf1, priority=9, startTime=1675954544810; duration=0sec 2023-02-09 14:55:44,858 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:55:44,859 DEBUG [StoreOpener-6c7521c64dfe564c431c1e4733b56ec3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/ef098f0b243849bc90a19fb43914356c.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/ef098f0b243849bc90a19fb43914356c-top 2023-02-09 14:55:44,859 INFO [StoreOpener-6c7521c64dfe564c431c1e4733b56ec3-1] regionserver.HStore(310): Store=6c7521c64dfe564c431c1e4733b56ec3/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:44,859 INFO [StoreOpener-6c7521c64dfe564c431c1e4733b56ec3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,860 DEBUG [StoreOpener-6c7521c64dfe564c431c1e4733b56ec3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2 2023-02-09 14:55:44,860 DEBUG [StoreOpener-6c7521c64dfe564c431c1e4733b56ec3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2 2023-02-09 14:55:44,860 INFO [StoreOpener-6c7521c64dfe564c431c1e4733b56ec3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6c7521c64dfe564c431c1e4733b56ec3 columnFamilyName cf2 2023-02-09 14:55:44,874 DEBUG [StoreOpener-6c7521c64dfe564c431c1e4733b56ec3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/333988a5c29b47449390ab71b1a3ef2f.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/333988a5c29b47449390ab71b1a3ef2f-top 2023-02-09 14:55:44,877 DEBUG [StoreOpener-6c7521c64dfe564c431c1e4733b56ec3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/bafbe116570d4368879cf0bbc41638d3.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/bafbe116570d4368879cf0bbc41638d3-top 2023-02-09 14:55:44,880 DEBUG [StoreOpener-6c7521c64dfe564c431c1e4733b56ec3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/d30ffcea033d4133941ec24b991abfbe.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/d30ffcea033d4133941ec24b991abfbe-top 2023-02-09 14:55:44,882 DEBUG [StoreOpener-6c7521c64dfe564c431c1e4733b56ec3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/e59ca37300bd48b6a4ee7ac39554cbaf.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e59ca37300bd48b6a4ee7ac39554cbaf-top 2023-02-09 14:55:44,885 DEBUG [StoreOpener-6c7521c64dfe564c431c1e4733b56ec3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/e8315fec79ba4d5ab09ccd57719ed2a6.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e8315fec79ba4d5ab09ccd57719ed2a6-top 2023-02-09 14:55:44,887 DEBUG [StoreOpener-6c7521c64dfe564c431c1e4733b56ec3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/f0401a668795474bba66cfc5da136faf.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/f0401a668795474bba66cfc5da136faf-top 2023-02-09 14:55:44,890 DEBUG [StoreOpener-6c7521c64dfe564c431c1e4733b56ec3-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/fdc7589cbab3484eb9c2b4473a70e219.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/fdc7589cbab3484eb9c2b4473a70e219-top 2023-02-09 14:55:44,890 INFO [StoreOpener-6c7521c64dfe564c431c1e4733b56ec3-1] regionserver.HStore(310): Store=6c7521c64dfe564c431c1e4733b56ec3/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:44,891 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,891 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,893 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:55:44,894 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 6c7521c64dfe564c431c1e4733b56ec3; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68958794, jitterRate=0.027566105127334595}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:55:44,894 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 6c7521c64dfe564c431c1e4733b56ec3: Running coprocessor pre-open hook at 1675954544816Writing region info on filesystem at 1675954544816Initializing all the Stores at 1675954544817 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954544817Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954544817Cleaning up temporary data from old regions at 1675954544892 (+75 ms)Cleaning up detritus from prior splits at 1675954544892Running coprocessor post-open hooks at 1675954544894 (+2 ms)Region opened successfully at 1675954544894 2023-02-09 14:55:44,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3., pid=118, masterSystemTime=1675954544737 2023-02-09 14:55:44,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:55:44,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 14:55:44,896 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 14:55:44,896 INFO [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3. 2023-02-09 14:55:44,896 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.HStore(1912): 6c7521c64dfe564c431c1e4733b56ec3/cf1 is initiating minor compaction (all files) 2023-02-09 14:55:44,896 INFO [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 6c7521c64dfe564c431c1e4733b56ec3/cf1 in testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3. 2023-02-09 14:55:44,897 INFO [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/a551eb3bbc834e1f80951082e085d1fd.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/a551eb3bbc834e1f80951082e085d1fd-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/d2e156e7ebf64b6ba510ea828773eca1.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/d2e156e7ebf64b6ba510ea828773eca1-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/5b999bec37ed41eca0c9366717969996.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/5b999bec37ed41eca0c9366717969996-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/09103225a5724a4d8c9313c7f68a607e.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/09103225a5724a4d8c9313c7f68a607e-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/8c0ffa708ab14279b63e065ccfd3ce98.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/8c0ffa708ab14279b63e065ccfd3ce98-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/ef098f0b243849bc90a19fb43914356c.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/ef098f0b243849bc90a19fb43914356c-top] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/.tmp, totalSize=34.9 K 2023-02-09 14:55:44,897 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] compactions.Compactor(207): Compacting e2b4de5742234f5a9ac8c98a9dfa7ae5.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675954539464 2023-02-09 14:55:44,897 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3. 2023-02-09 14:55:44,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3. 2023-02-09 14:55:44,897 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] compactions.Compactor(207): Compacting a551eb3bbc834e1f80951082e085d1fd.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675954539631 2023-02-09 14:55:44,898 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=6c7521c64dfe564c431c1e4733b56ec3, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:44,898 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] compactions.Compactor(207): Compacting d2e156e7ebf64b6ba510ea828773eca1.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675954539804 2023-02-09 14:55:44,898 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675954544898"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954544898"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954544898"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954544898"}]},"ts":"1675954544898"} 2023-02-09 14:55:44,898 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] compactions.Compactor(207): Compacting 5b999bec37ed41eca0c9366717969996.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675954539988 2023-02-09 14:55:44,899 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] compactions.Compactor(207): Compacting 09103225a5724a4d8c9313c7f68a607e.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675954540159 2023-02-09 14:55:44,899 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] compactions.Compactor(207): Compacting 8c0ffa708ab14279b63e065ccfd3ce98.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675954540330 2023-02-09 14:55:44,899 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] compactions.Compactor(207): Compacting ef098f0b243849bc90a19fb43914356c.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675954540496 2023-02-09 14:55:44,900 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=116 2023-02-09 14:55:44,900 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 6c7521c64dfe564c431c1e4733b56ec3, server=jenkins-hbase4.apache.org,33553,1675954443180 in 311 msec 2023-02-09 14:55:44,902 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=116, resume processing ppid=112 2023-02-09 14:55:44,902 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6c7521c64dfe564c431c1e4733b56ec3, ASSIGN in 471 msec 2023-02-09 14:55:44,902 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-09 14:55:44,902 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,902 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:55:44,902 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-09 14:55:44,908 INFO [RS:1;jenkins-hbase4:33553-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6c7521c64dfe564c431c1e4733b56ec3#cf1#compaction#110 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:55:44,917 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/.tmp/cf1/eb351a9e1004424cab452824fba3f473 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/eb351a9e1004424cab452824fba3f473 2023-02-09 14:55:44,921 INFO [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6c7521c64dfe564c431c1e4733b56ec3/cf1 of 6c7521c64dfe564c431c1e4733b56ec3 into eb351a9e1004424cab452824fba3f473(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:55:44,921 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6c7521c64dfe564c431c1e4733b56ec3: 2023-02-09 14:55:44,921 INFO [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3., storeName=6c7521c64dfe564c431c1e4733b56ec3/cf1, priority=9, startTime=1675954544895; duration=0sec 2023-02-09 14:55:44,921 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 14:55:44,921 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 14:55:44,921 INFO [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3. 2023-02-09 14:55:44,921 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.HStore(1912): 6c7521c64dfe564c431c1e4733b56ec3/cf2 is initiating minor compaction (all files) 2023-02-09 14:55:44,922 INFO [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 6c7521c64dfe564c431c1e4733b56ec3/cf2 in testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3. 2023-02-09 14:55:44,922 INFO [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/e59ca37300bd48b6a4ee7ac39554cbaf.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e59ca37300bd48b6a4ee7ac39554cbaf-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/d30ffcea033d4133941ec24b991abfbe.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/d30ffcea033d4133941ec24b991abfbe-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/333988a5c29b47449390ab71b1a3ef2f.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/333988a5c29b47449390ab71b1a3ef2f-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/fdc7589cbab3484eb9c2b4473a70e219.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/fdc7589cbab3484eb9c2b4473a70e219-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/bafbe116570d4368879cf0bbc41638d3.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/bafbe116570d4368879cf0bbc41638d3-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/e8315fec79ba4d5ab09ccd57719ed2a6.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e8315fec79ba4d5ab09ccd57719ed2a6-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/f0401a668795474bba66cfc5da136faf.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/f0401a668795474bba66cfc5da136faf-top] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/.tmp, totalSize=34.9 K 2023-02-09 14:55:44,922 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] compactions.Compactor(207): Compacting e59ca37300bd48b6a4ee7ac39554cbaf.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675954539464 2023-02-09 14:55:44,923 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] compactions.Compactor(207): Compacting d30ffcea033d4133941ec24b991abfbe.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675954539631 2023-02-09 14:55:44,923 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] compactions.Compactor(207): Compacting 333988a5c29b47449390ab71b1a3ef2f.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675954539804 2023-02-09 14:55:44,923 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] compactions.Compactor(207): Compacting fdc7589cbab3484eb9c2b4473a70e219.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675954539988 2023-02-09 14:55:44,924 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] compactions.Compactor(207): Compacting bafbe116570d4368879cf0bbc41638d3.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675954540159 2023-02-09 14:55:44,924 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] compactions.Compactor(207): Compacting e8315fec79ba4d5ab09ccd57719ed2a6.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675954540330 2023-02-09 14:55:44,924 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] compactions.Compactor(207): Compacting f0401a668795474bba66cfc5da136faf.a38bfbdf613fc88c3b7b74edf1f8bd28, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675954540496 2023-02-09 14:55:44,930 INFO [Listener at localhost/40823] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,930 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-09 14:55:44,930 INFO [Listener at localhost/40823] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:55:44,930 INFO [Listener at localhost/40823] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:55:44,930 ERROR [Listener at localhost/40823] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-09 14:55:44,931 INFO [Listener at localhost/40823] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-09 14:55:44,931 INFO [Listener at localhost/40823] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-09 14:55:44,931 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-09 14:55:44,931 INFO [Listener at localhost/40823] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-09 14:55:44,931 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-09 14:55:44,932 INFO [RS:1;jenkins-hbase4:33553-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6c7521c64dfe564c431c1e4733b56ec3#cf2#compaction#111 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:55:44,933 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-02-09 14:55:44,933 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-09 14:55:44,933 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-09 14:55:44,933 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 16, 21] 2023-02-09 14:55:44,933 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 20] 2023-02-09 14:55:44,933 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 18, 23] 2023-02-09 14:55:44,934 DEBUG [Listener at localhost/40823] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 22] 2023-02-09 14:55:44,934 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-09 14:55:44,934 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-09 14:55:44,934 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-09 14:55:44,935 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-09 14:55:44,935 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:44,935 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=d88a751f86ea75609ea8bbee5604986e, daughterA=6a1146514f3e0fd1da817878786a40fe, daughterB=e91d34f46fc227e8f7ee0e968792274e 2023-02-09 14:55:44,935 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:44,935 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-09 14:55:44,935 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:44,936 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=2f41522f4f390d8b073e3a5237c42fff, daughterA=f9cd34e6dad51e106afa9855e7d82622, daughterB=10ba4d60d6f9660fb4081f6b504506d4 2023-02-09 14:55:44,936 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:44,936 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-09 14:55:44,936 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:44,936 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=458ec544613af51e1d69337b4a7fa39b, daughterA=485d653fadb7b7df523ff9ae46d1b78c, daughterB=d4498432e15515d5f43c5d2932f1cffa 2023-02-09 14:55:44,937 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=458ec544613af51e1d69337b4a7fa39b, ASSIGN 2023-02-09 14:55:44,937 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:44,937 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-09 14:55:44,937 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:44,937 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=7a3a2a2c1be8b68cf6b9c5576ca1d967, daughterA=915612316d6a09fe13e83a17e2d06051, daughterB=2c62e1c084902d2aa62c5102c5ff76cd 2023-02-09 14:55:44,937 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:44,937 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-09 14:55:44,937 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:44,938 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=94ffc82cbf109e7f71870e19837a0e78, daughterA=c720e43e6c74641ba50283474b45a523, daughterB=8c23d3f07541daba404c2b3d3332d0cb 2023-02-09 14:55:44,938 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:44,938 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-09 14:55:44,938 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:44,938 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:44,938 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,41077,1675954443212, table=testRollbackForSplitTableRegionWithReplica, region=813d8870e2a4f2929f4991fa0f505d12 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=080674692710b4dfd3481fd1d48b1c60, daughterA=45103f245599b3c2b8e9be2a40c7083b, daughterB=47b175695c3d18e949263cbc873d2e48 2023-02-09 14:55:44,938 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:44,938 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-09 14:55:44,938 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:44,939 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:44,939 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-09 14:55:44,939 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 14:55:44,940 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,940 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28, UNASSIGN 2023-02-09 14:55:44,940 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a38bfbdf613fc88c3b7b74edf1f8bd28, server=jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:44,940 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8c791f527032c6c0e52c041969bac8e3, ASSIGN 2023-02-09 14:55:44,940 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6c7521c64dfe564c431c1e4733b56ec3, ASSIGN 2023-02-09 14:55:44,940 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 8c791f527032c6c0e52c041969bac8e3, server=jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:44,940 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 6c7521c64dfe564c431c1e4733b56ec3, server=jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:44,941 DEBUG [Listener at localhost/40823] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 held the lock before restarting, call acquireLock to restore it. 2023-02-09 14:55:44,941 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,941 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,941 INFO [Listener at localhost/40823] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,941 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:55:44,941 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure a38bfbdf613fc88c3b7b74edf1f8bd28, server=jenkins-hbase4.apache.org,33553,1675954443180 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:55:44,941 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8c791f527032c6c0e52c041969bac8e3, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:55:44,941 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6c7521c64dfe564c431c1e4733b56ec3, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:55:44,941 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 8c791f527032c6c0e52c041969bac8e3, server=jenkins-hbase4.apache.org,33553,1675954443180 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:55:44,941 DEBUG [Listener at localhost/40823] procedure2.Procedure(946): pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 6c7521c64dfe564c431c1e4733b56ec3, server=jenkins-hbase4.apache.org,33553,1675954443180 didn't hold the lock before restarting, skip acquiring lock. 2023-02-09 14:55:44,941 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-09 14:55:44,944 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-09 14:55:44,945 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133, table=hbase:meta, region=1588230740 2023-02-09 14:55:44,947 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/.tmp/cf2/d67048b39611490f8370a7dec035b22c as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/d67048b39611490f8370a7dec035b22c 2023-02-09 14:55:44,948 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675954574948 2023-02-09 14:55:44,950 DEBUG [Listener at localhost/40823] assignment.AssignmentManager(1519): Joining cluster... 2023-02-09 14:55:44,950 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:44,952 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=0c1f8115bde7013387f9461b40485fe1, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33423,1675954443133, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133, openSeqNum=2 2023-02-09 14:55:44,953 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=a38bfbdf613fc88c3b7b74edf1f8bd28, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,33553,1675954443180, regionLocation=jenkins-hbase4.apache.org,33553,1675954443180, openSeqNum=2 2023-02-09 14:55:44,953 INFO [Listener at localhost/40823] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,33553,1675954443180, table=testRecoveryAndDoubleExecution, region=a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:44,953 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=8c791f527032c6c0e52c041969bac8e3, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33553,1675954443180, regionLocation=jenkins-hbase4.apache.org,33553,1675954443180, openSeqNum=86 2023-02-09 14:55:44,953 DEBUG [Listener at localhost/40823] assignment.RegionStateStore(147): Load hbase:meta entry region=6c7521c64dfe564c431c1e4733b56ec3, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,33553,1675954443180, regionLocation=jenkins-hbase4.apache.org,33553,1675954443180, openSeqNum=86 2023-02-09 14:55:44,953 INFO [Listener at localhost/40823] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-09 14:55:44,953 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675954604953 2023-02-09 14:55:44,953 INFO [Listener at localhost/40823] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675954664953 2023-02-09 14:55:44,953 INFO [Listener at localhost/40823] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-09 14:55:44,954 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-09 14:55:44,955 INFO [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6c7521c64dfe564c431c1e4733b56ec3/cf2 of 6c7521c64dfe564c431c1e4733b56ec3 into d67048b39611490f8370a7dec035b22c(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:55:44,955 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6c7521c64dfe564c431c1e4733b56ec3: 2023-02-09 14:55:44,955 INFO [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3., storeName=6c7521c64dfe564c431c1e4733b56ec3/cf2, priority=9, startTime=1675954544895; duration=0sec 2023-02-09 14:55:44,955 DEBUG [RS:1;jenkins-hbase4:33553-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:55:44,956 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=a38bfbdf613fc88c3b7b74edf1f8bd28, daughterA=8c791f527032c6c0e52c041969bac8e3, daughterB=6c7521c64dfe564c431c1e4733b56ec3 in 4.2000 sec 2023-02-09 14:55:45,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33553] ipc.CallRunner(144): callId: 856 service: ClientService methodName: Scan size: 154 connection: 172.31.14.131:34150 deadline: 1675954605204, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28. is not online on jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:45,255 DEBUG [RS:1;jenkins-hbase4:33553-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/.tmp/cf2/17a68adb64be497fb64a3b6757966c62 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/17a68adb64be497fb64a3b6757966c62 2023-02-09 14:55:45,259 INFO [RS:1;jenkins-hbase4:33553-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 8c791f527032c6c0e52c041969bac8e3/cf2 of 8c791f527032c6c0e52c041969bac8e3 into 17a68adb64be497fb64a3b6757966c62(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:55:45,259 DEBUG [RS:1;jenkins-hbase4:33553-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 8c791f527032c6c0e52c041969bac8e3: 2023-02-09 14:55:45,259 INFO [RS:1;jenkins-hbase4:33553-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3., storeName=8c791f527032c6c0e52c041969bac8e3/cf2, priority=9, startTime=1675954544810; duration=0sec 2023-02-09 14:55:45,259 DEBUG [RS:1;jenkins-hbase4:33553-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:55:45,315 DEBUG [Listener at localhost/40823] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 14:55:45,321 INFO [Listener at localhost/40823] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-09 14:55:45,322 DEBUG [Listener at localhost/40823] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 14:55:45,329 WARN [Listener at localhost/40823] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 14:55:45,329 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:45,329 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:45,330 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:45,330 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:45,331 INFO [Listener at localhost/40823] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-02-09 14:55:45,331 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testRecoveryAndDoubleExecution 2023-02-09 14:55:45,331 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 14:55:45,333 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:45,333 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:45,334 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-09 14:55:45,334 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954545334"}]},"ts":"1675954545334"} 2023-02-09 14:55:45,335 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-09 14:55:45,336 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-02-09 14:55:45,336 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8c791f527032c6c0e52c041969bac8e3, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6c7521c64dfe564c431c1e4733b56ec3, UNASSIGN}] 2023-02-09 14:55:45,337 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6c7521c64dfe564c431c1e4733b56ec3, UNASSIGN 2023-02-09 14:55:45,337 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=6c7521c64dfe564c431c1e4733b56ec3, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:45,337 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675954545337"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954545337"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954545337"}]},"ts":"1675954545337"} 2023-02-09 14:55:45,338 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure 6c7521c64dfe564c431c1e4733b56ec3, server=jenkins-hbase4.apache.org,33553,1675954443180}] 2023-02-09 14:55:45,340 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8c791f527032c6c0e52c041969bac8e3, UNASSIGN 2023-02-09 14:55:45,340 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=8c791f527032c6c0e52c041969bac8e3, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:45,341 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675954545340"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954545340"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954545340"}]},"ts":"1675954545340"} 2023-02-09 14:55:45,341 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure 8c791f527032c6c0e52c041969bac8e3, server=jenkins-hbase4.apache.org,33553,1675954443180}] 2023-02-09 14:55:45,434 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:45,434 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:45,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-09 14:55:45,490 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:45,490 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 6c7521c64dfe564c431c1e4733b56ec3, disabling compactions & flushes 2023-02-09 14:55:45,490 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3. 2023-02-09 14:55:45,490 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3. 2023-02-09 14:55:45,490 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3. after waiting 0 ms 2023-02-09 14:55:45,490 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3. 2023-02-09 14:55:45,491 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/a551eb3bbc834e1f80951082e085d1fd.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/a551eb3bbc834e1f80951082e085d1fd-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/d2e156e7ebf64b6ba510ea828773eca1.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/d2e156e7ebf64b6ba510ea828773eca1-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/5b999bec37ed41eca0c9366717969996.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/5b999bec37ed41eca0c9366717969996-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/09103225a5724a4d8c9313c7f68a607e.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/09103225a5724a4d8c9313c7f68a607e-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/8c0ffa708ab14279b63e065ccfd3ce98.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/8c0ffa708ab14279b63e065ccfd3ce98-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/ef098f0b243849bc90a19fb43914356c.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/ef098f0b243849bc90a19fb43914356c-top] to archive 2023-02-09 14:55:45,491 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:45,493 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,494 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/a551eb3bbc834e1f80951082e085d1fd.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/a551eb3bbc834e1f80951082e085d1fd.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,495 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/d2e156e7ebf64b6ba510ea828773eca1.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/d2e156e7ebf64b6ba510ea828773eca1.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,496 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/5b999bec37ed41eca0c9366717969996.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/5b999bec37ed41eca0c9366717969996.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,497 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/09103225a5724a4d8c9313c7f68a607e.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/09103225a5724a4d8c9313c7f68a607e.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,497 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/8c0ffa708ab14279b63e065ccfd3ce98.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/8c0ffa708ab14279b63e065ccfd3ce98.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,498 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/ef098f0b243849bc90a19fb43914356c.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/ef098f0b243849bc90a19fb43914356c.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,500 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/e59ca37300bd48b6a4ee7ac39554cbaf.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e59ca37300bd48b6a4ee7ac39554cbaf-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/d30ffcea033d4133941ec24b991abfbe.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/d30ffcea033d4133941ec24b991abfbe-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/333988a5c29b47449390ab71b1a3ef2f.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/333988a5c29b47449390ab71b1a3ef2f-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/fdc7589cbab3484eb9c2b4473a70e219.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/fdc7589cbab3484eb9c2b4473a70e219-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/bafbe116570d4368879cf0bbc41638d3.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/bafbe116570d4368879cf0bbc41638d3-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/e8315fec79ba4d5ab09ccd57719ed2a6.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e8315fec79ba4d5ab09ccd57719ed2a6-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/f0401a668795474bba66cfc5da136faf.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/f0401a668795474bba66cfc5da136faf-top] to archive 2023-02-09 14:55:45,500 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:45,501 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/e59ca37300bd48b6a4ee7ac39554cbaf.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/e59ca37300bd48b6a4ee7ac39554cbaf.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,502 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/d30ffcea033d4133941ec24b991abfbe.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/d30ffcea033d4133941ec24b991abfbe.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,503 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/333988a5c29b47449390ab71b1a3ef2f.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/333988a5c29b47449390ab71b1a3ef2f.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,504 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/fdc7589cbab3484eb9c2b4473a70e219.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/fdc7589cbab3484eb9c2b4473a70e219.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,505 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/bafbe116570d4368879cf0bbc41638d3.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/bafbe116570d4368879cf0bbc41638d3.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,506 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/e8315fec79ba4d5ab09ccd57719ed2a6.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/e8315fec79ba4d5ab09ccd57719ed2a6.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,507 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/f0401a668795474bba66cfc5da136faf.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/f0401a668795474bba66cfc5da136faf.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,510 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 14:55:45,511 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:55:45,511 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3. 2023-02-09 14:55:45,511 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 6c7521c64dfe564c431c1e4733b56ec3: Waiting for close lock at 1675954545490Running coprocessor pre-close hooks at 1675954545490Disabling compacts and flushes for region at 1675954545490Disabling writes for close at 1675954545490Writing region close event to WAL at 1675954545508 (+18 ms)Running coprocessor post-close hooks at 1675954545511 (+3 ms)Closed at 1675954545511 2023-02-09 14:55:45,513 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:45,513 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 8c791f527032c6c0e52c041969bac8e3 2023-02-09 14:55:45,513 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 8c791f527032c6c0e52c041969bac8e3, disabling compactions & flushes 2023-02-09 14:55:45,513 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3. 2023-02-09 14:55:45,513 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3. 2023-02-09 14:55:45,513 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3. after waiting 0 ms 2023-02-09 14:55:45,513 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3. 2023-02-09 14:55:45,513 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=6c7521c64dfe564c431c1e4733b56ec3, regionState=CLOSED 2023-02-09 14:55:45,514 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675954545513"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954545513"}]},"ts":"1675954545513"} 2023-02-09 14:55:45,515 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/a551eb3bbc834e1f80951082e085d1fd.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/a551eb3bbc834e1f80951082e085d1fd-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/d2e156e7ebf64b6ba510ea828773eca1.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/d2e156e7ebf64b6ba510ea828773eca1-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/5b999bec37ed41eca0c9366717969996.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/5b999bec37ed41eca0c9366717969996-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/09103225a5724a4d8c9313c7f68a607e.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/09103225a5724a4d8c9313c7f68a607e-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/8c0ffa708ab14279b63e065ccfd3ce98.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/8c0ffa708ab14279b63e065ccfd3ce98-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/ef098f0b243849bc90a19fb43914356c.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/ef098f0b243849bc90a19fb43914356c-bottom] to archive 2023-02-09 14:55:45,515 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:45,516 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-02-09 14:55:45,516 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure 6c7521c64dfe564c431c1e4733b56ec3, server=jenkins-hbase4.apache.org,33553,1675954443180 in 177 msec 2023-02-09 14:55:45,517 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,517 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6c7521c64dfe564c431c1e4733b56ec3, UNASSIGN in 180 msec 2023-02-09 14:55:45,518 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/a551eb3bbc834e1f80951082e085d1fd.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/a551eb3bbc834e1f80951082e085d1fd.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,519 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/d2e156e7ebf64b6ba510ea828773eca1.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/d2e156e7ebf64b6ba510ea828773eca1.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,520 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/5b999bec37ed41eca0c9366717969996.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/5b999bec37ed41eca0c9366717969996.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,520 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/09103225a5724a4d8c9313c7f68a607e.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/09103225a5724a4d8c9313c7f68a607e.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,521 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/8c0ffa708ab14279b63e065ccfd3ce98.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/8c0ffa708ab14279b63e065ccfd3ce98.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,522 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/ef098f0b243849bc90a19fb43914356c.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/ef098f0b243849bc90a19fb43914356c.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,523 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/e59ca37300bd48b6a4ee7ac39554cbaf.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e59ca37300bd48b6a4ee7ac39554cbaf-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/d30ffcea033d4133941ec24b991abfbe.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/d30ffcea033d4133941ec24b991abfbe-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/333988a5c29b47449390ab71b1a3ef2f.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/333988a5c29b47449390ab71b1a3ef2f-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/fdc7589cbab3484eb9c2b4473a70e219.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/fdc7589cbab3484eb9c2b4473a70e219-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/bafbe116570d4368879cf0bbc41638d3.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/bafbe116570d4368879cf0bbc41638d3-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/e8315fec79ba4d5ab09ccd57719ed2a6.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e8315fec79ba4d5ab09ccd57719ed2a6-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/f0401a668795474bba66cfc5da136faf.a38bfbdf613fc88c3b7b74edf1f8bd28->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/f0401a668795474bba66cfc5da136faf-bottom] to archive 2023-02-09 14:55:45,524 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:45,525 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/e59ca37300bd48b6a4ee7ac39554cbaf.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/e59ca37300bd48b6a4ee7ac39554cbaf.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,526 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/d30ffcea033d4133941ec24b991abfbe.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/d30ffcea033d4133941ec24b991abfbe.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,527 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/333988a5c29b47449390ab71b1a3ef2f.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/333988a5c29b47449390ab71b1a3ef2f.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,527 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/fdc7589cbab3484eb9c2b4473a70e219.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/fdc7589cbab3484eb9c2b4473a70e219.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,528 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/bafbe116570d4368879cf0bbc41638d3.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/bafbe116570d4368879cf0bbc41638d3.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,529 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/e8315fec79ba4d5ab09ccd57719ed2a6.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/e8315fec79ba4d5ab09ccd57719ed2a6.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,530 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/f0401a668795474bba66cfc5da136faf.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/f0401a668795474bba66cfc5da136faf.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,533 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 14:55:45,533 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:55:45,533 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3. 2023-02-09 14:55:45,533 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 8c791f527032c6c0e52c041969bac8e3: Waiting for close lock at 1675954545513Running coprocessor pre-close hooks at 1675954545513Disabling compacts and flushes for region at 1675954545513Disabling writes for close at 1675954545513Writing region close event to WAL at 1675954545531 (+18 ms)Running coprocessor post-close hooks at 1675954545533 (+2 ms)Closed at 1675954545533 2023-02-09 14:55:45,535 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 8c791f527032c6c0e52c041969bac8e3 2023-02-09 14:55:45,535 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=8c791f527032c6c0e52c041969bac8e3, regionState=CLOSED 2023-02-09 14:55:45,536 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675954545535"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954545535"}]},"ts":"1675954545535"} 2023-02-09 14:55:45,537 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-02-09 14:55:45,537 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure 8c791f527032c6c0e52c041969bac8e3, server=jenkins-hbase4.apache.org,33553,1675954443180 in 195 msec 2023-02-09 14:55:45,539 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-02-09 14:55:45,539 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8c791f527032c6c0e52c041969bac8e3, UNASSIGN in 202 msec 2023-02-09 14:55:45,539 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954545539"}]},"ts":"1675954545539"} 2023-02-09 14:55:45,540 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-09 14:55:45,541 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-02-09 14:55:45,542 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 211 msec 2023-02-09 14:55:45,635 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:45,635 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:45,636 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-09 14:55:45,636 INFO [Listener at localhost/40823] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-02-09 14:55:45,636 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:45,636 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:45,637 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testRecoveryAndDoubleExecution 2023-02-09 14:55:45,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 14:55:45,639 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 14:55:45,639 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:45,639 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:45,639 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 14:55:45,640 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-09 14:55:45,644 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,644 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3 2023-02-09 14:55:45,644 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:45,646 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/recovered.edits] 2023-02-09 14:55:45,646 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/recovered.edits] 2023-02-09 14:55:45,646 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/recovered.edits] 2023-02-09 14:55:45,651 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/.regioninfo to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/.regioninfo 2023-02-09 14:55:45,651 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/.regioninfo to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/.regioninfo 2023-02-09 14:55:45,653 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/09103225a5724a4d8c9313c7f68a607e to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/09103225a5724a4d8c9313c7f68a607e 2023-02-09 14:55:45,656 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/5b999bec37ed41eca0c9366717969996 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/5b999bec37ed41eca0c9366717969996 2023-02-09 14:55:45,656 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf1/09103225a5724a4d8c9313c7f68a607e.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf1/09103225a5724a4d8c9313c7f68a607e.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,657 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf1/09103225a5724a4d8c9313c7f68a607e.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf1/09103225a5724a4d8c9313c7f68a607e.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,657 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/8c0ffa708ab14279b63e065ccfd3ce98 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/8c0ffa708ab14279b63e065ccfd3ce98 2023-02-09 14:55:45,657 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf1/5b999bec37ed41eca0c9366717969996.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf1/5b999bec37ed41eca0c9366717969996.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,658 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf1/5b999bec37ed41eca0c9366717969996.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf1/5b999bec37ed41eca0c9366717969996.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,659 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf1/8c0ffa708ab14279b63e065ccfd3ce98.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf1/8c0ffa708ab14279b63e065ccfd3ce98.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,659 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/a551eb3bbc834e1f80951082e085d1fd to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/a551eb3bbc834e1f80951082e085d1fd 2023-02-09 14:55:45,660 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf1/8c0ffa708ab14279b63e065ccfd3ce98.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf1/8c0ffa708ab14279b63e065ccfd3ce98.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,660 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf1/a551eb3bbc834e1f80951082e085d1fd.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf1/a551eb3bbc834e1f80951082e085d1fd.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,661 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/d2e156e7ebf64b6ba510ea828773eca1 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/d2e156e7ebf64b6ba510ea828773eca1 2023-02-09 14:55:45,661 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf1/a551eb3bbc834e1f80951082e085d1fd.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf1/a551eb3bbc834e1f80951082e085d1fd.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,662 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf1/d2e156e7ebf64b6ba510ea828773eca1.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf1/d2e156e7ebf64b6ba510ea828773eca1.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,662 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5 2023-02-09 14:55:45,663 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf1/d2e156e7ebf64b6ba510ea828773eca1.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf1/d2e156e7ebf64b6ba510ea828773eca1.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,663 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,663 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/ef098f0b243849bc90a19fb43914356c to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf1/ef098f0b243849bc90a19fb43914356c 2023-02-09 14:55:45,664 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf1/e2b4de5742234f5a9ac8c98a9dfa7ae5.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,664 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf1/ef098f0b243849bc90a19fb43914356c.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf1/ef098f0b243849bc90a19fb43914356c.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,665 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf1/ef098f0b243849bc90a19fb43914356c.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf1/ef098f0b243849bc90a19fb43914356c.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,667 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/333988a5c29b47449390ab71b1a3ef2f to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/333988a5c29b47449390ab71b1a3ef2f 2023-02-09 14:55:45,668 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf2/333988a5c29b47449390ab71b1a3ef2f.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf2/333988a5c29b47449390ab71b1a3ef2f.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,668 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/bafbe116570d4368879cf0bbc41638d3 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/bafbe116570d4368879cf0bbc41638d3 2023-02-09 14:55:45,669 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf2/333988a5c29b47449390ab71b1a3ef2f.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf2/333988a5c29b47449390ab71b1a3ef2f.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,669 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf2/bafbe116570d4368879cf0bbc41638d3.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf2/bafbe116570d4368879cf0bbc41638d3.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,670 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/d30ffcea033d4133941ec24b991abfbe to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/d30ffcea033d4133941ec24b991abfbe 2023-02-09 14:55:45,670 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf2/bafbe116570d4368879cf0bbc41638d3.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf2/bafbe116570d4368879cf0bbc41638d3.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,671 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf2/d30ffcea033d4133941ec24b991abfbe.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf2/d30ffcea033d4133941ec24b991abfbe.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,671 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e59ca37300bd48b6a4ee7ac39554cbaf to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e59ca37300bd48b6a4ee7ac39554cbaf 2023-02-09 14:55:45,671 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf2/d30ffcea033d4133941ec24b991abfbe.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf2/d30ffcea033d4133941ec24b991abfbe.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,672 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf2/e59ca37300bd48b6a4ee7ac39554cbaf.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf2/e59ca37300bd48b6a4ee7ac39554cbaf.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,673 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e8315fec79ba4d5ab09ccd57719ed2a6 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/e8315fec79ba4d5ab09ccd57719ed2a6 2023-02-09 14:55:45,673 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf2/e59ca37300bd48b6a4ee7ac39554cbaf.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf2/e59ca37300bd48b6a4ee7ac39554cbaf.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,673 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf2/e8315fec79ba4d5ab09ccd57719ed2a6.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf2/e8315fec79ba4d5ab09ccd57719ed2a6.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,674 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/f0401a668795474bba66cfc5da136faf to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/f0401a668795474bba66cfc5da136faf 2023-02-09 14:55:45,674 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf2/e8315fec79ba4d5ab09ccd57719ed2a6.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf2/e8315fec79ba4d5ab09ccd57719ed2a6.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,675 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf2/f0401a668795474bba66cfc5da136faf.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf2/f0401a668795474bba66cfc5da136faf.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,676 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/fdc7589cbab3484eb9c2b4473a70e219 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/cf2/fdc7589cbab3484eb9c2b4473a70e219 2023-02-09 14:55:45,676 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf2/f0401a668795474bba66cfc5da136faf.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf2/f0401a668795474bba66cfc5da136faf.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,676 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf2/fdc7589cbab3484eb9c2b4473a70e219.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/6c7521c64dfe564c431c1e4733b56ec3/cf2/fdc7589cbab3484eb9c2b4473a70e219.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,678 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf2/fdc7589cbab3484eb9c2b4473a70e219.a38bfbdf613fc88c3b7b74edf1f8bd28 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/8c791f527032c6c0e52c041969bac8e3/cf2/fdc7589cbab3484eb9c2b4473a70e219.a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,679 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/recovered.edits/85.seqid to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28/recovered.edits/85.seqid 2023-02-09 14:55:45,680 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/eb351a9e1004424cab452824fba3f473 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf1/eb351a9e1004424cab452824fba3f473 2023-02-09 14:55:45,680 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/a38bfbdf613fc88c3b7b74edf1f8bd28 2023-02-09 14:55:45,680 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/e381142197cc48b8a18476236bcce5d5 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf1/e381142197cc48b8a18476236bcce5d5 2023-02-09 14:55:45,684 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/d67048b39611490f8370a7dec035b22c to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/cf2/d67048b39611490f8370a7dec035b22c 2023-02-09 14:55:45,684 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/17a68adb64be497fb64a3b6757966c62 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/cf2/17a68adb64be497fb64a3b6757966c62 2023-02-09 14:55:45,687 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/recovered.edits/90.seqid to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3/recovered.edits/90.seqid 2023-02-09 14:55:45,687 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/recovered.edits/90.seqid to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3/recovered.edits/90.seqid 2023-02-09 14:55:45,687 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/8c791f527032c6c0e52c041969bac8e3 2023-02-09 14:55:45,688 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testRecoveryAndDoubleExecution/6c7521c64dfe564c431c1e4733b56ec3 2023-02-09 14:55:45,688 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-09 14:55:45,689 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 14:55:45,691 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-02-09 14:55:45,693 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-02-09 14:55:45,693 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 14:55:45,693 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-02-09 14:55:45,694 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954545694"}]},"ts":"9223372036854775807"} 2023-02-09 14:55:45,694 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954545694"}]},"ts":"9223372036854775807"} 2023-02-09 14:55:45,694 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954545694"}]},"ts":"9223372036854775807"} 2023-02-09 14:55:45,695 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 14:55:45,695 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => a38bfbdf613fc88c3b7b74edf1f8bd28, NAME => 'testRecoveryAndDoubleExecution,,1675954538955.a38bfbdf613fc88c3b7b74edf1f8bd28.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => 8c791f527032c6c0e52c041969bac8e3, NAME => 'testRecoveryAndDoubleExecution,,1675954540754.8c791f527032c6c0e52c041969bac8e3.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 6c7521c64dfe564c431c1e4733b56ec3, NAME => 'testRecoveryAndDoubleExecution,41,1675954540754.6c7521c64dfe564c431c1e4733b56ec3.', STARTKEY => '41', ENDKEY => ''}] 2023-02-09 14:55:45,695 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-02-09 14:55:45,695 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675954545695"}]},"ts":"9223372036854775807"} 2023-02-09 14:55:45,696 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-02-09 14:55:45,698 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-09 14:55:45,698 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 61 msec 2023-02-09 14:55:45,740 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:45,740 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:45,741 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-09 14:55:45,741 INFO [Listener at localhost/40823] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-02-09 14:55:45,756 INFO [Listener at localhost/40823] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=375 (was 362) - Thread LEAK? -, OpenFileDescriptor=833 (was 736) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=85 (was 91), ProcessCount=168 (was 168), AvailableMemoryMB=2946 (was 2977) 2023-02-09 14:55:45,769 INFO [Listener at localhost/40823] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=375, OpenFileDescriptor=833, MaxFileDescriptor=60000, SystemLoadAverage=85, ProcessCount=168, AvailableMemoryMB=2946 2023-02-09 14:55:45,769 WARN [Listener at localhost/40823] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 14:55:45,769 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:45,769 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:45,772 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 14:55:45,772 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 14:55:45,773 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 14:55:45,774 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 14:55:45,775 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 14:55:45,776 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:45,776 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd empty. 2023-02-09 14:55:45,777 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:45,777 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-09 14:55:45,785 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-02-09 14:55:45,786 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 109e2ee6b532fbae23a8a99627ce82fd, NAME => 'testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp 2023-02-09 14:55:45,792 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:45,792 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1603): Closing 109e2ee6b532fbae23a8a99627ce82fd, disabling compactions & flushes 2023-02-09 14:55:45,792 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. 2023-02-09 14:55:45,792 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. 2023-02-09 14:55:45,792 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. after waiting 0 ms 2023-02-09 14:55:45,792 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. 2023-02-09 14:55:45,792 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. 2023-02-09 14:55:45,792 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 109e2ee6b532fbae23a8a99627ce82fd: Waiting for close lock at 1675954545792Disabling compacts and flushes for region at 1675954545792Disabling writes for close at 1675954545792Writing region close event to WAL at 1675954545792Closed at 1675954545792 2023-02-09 14:55:45,794 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 14:55:45,794 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675954545794"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954545794"}]},"ts":"1675954545794"} 2023-02-09 14:55:45,795 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 14:55:45,796 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 14:55:45,796 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954545796"}]},"ts":"1675954545796"} 2023-02-09 14:55:45,797 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-02-09 14:55:45,799 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 14:55:45,800 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 14:55:45,800 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 14:55:45,800 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 14:55:45,800 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 14:55:45,800 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=109e2ee6b532fbae23a8a99627ce82fd, ASSIGN}] 2023-02-09 14:55:45,801 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=109e2ee6b532fbae23a8a99627ce82fd, ASSIGN 2023-02-09 14:55:45,801 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=109e2ee6b532fbae23a8a99627ce82fd, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33423,1675954443133; forceNewPlan=false, retain=false 2023-02-09 14:55:45,865 ERROR [BP-2069929948-172.31.14.131-1675954438937 heartbeating to localhost/127.0.0.1:38049] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2069929948-172.31.14.131-1675954438937 (Datanode Uuid 89217ea8-d9db-4980-9a81-2c27c9377629) service to localhost/127.0.0.1:38049 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-09 14:55:45,952 INFO [jenkins-hbase4:43829] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 14:55:45,953 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=109e2ee6b532fbae23a8a99627ce82fd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:45,953 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675954545953"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954545953"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954545953"}]},"ts":"1675954545953"} 2023-02-09 14:55:45,954 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 109e2ee6b532fbae23a8a99627ce82fd, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:55:46,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. 2023-02-09 14:55:46,108 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 109e2ee6b532fbae23a8a99627ce82fd, NAME => 'testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:55:46,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:55:46,108 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:46,108 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:46,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:46,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:46,110 INFO [StoreOpener-109e2ee6b532fbae23a8a99627ce82fd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:46,111 DEBUG [StoreOpener-109e2ee6b532fbae23a8a99627ce82fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1 2023-02-09 14:55:46,111 DEBUG [StoreOpener-109e2ee6b532fbae23a8a99627ce82fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1 2023-02-09 14:55:46,111 INFO [StoreOpener-109e2ee6b532fbae23a8a99627ce82fd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 109e2ee6b532fbae23a8a99627ce82fd columnFamilyName cf1 2023-02-09 14:55:46,111 INFO [StoreOpener-109e2ee6b532fbae23a8a99627ce82fd-1] regionserver.HStore(310): Store=109e2ee6b532fbae23a8a99627ce82fd/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:46,112 INFO [StoreOpener-109e2ee6b532fbae23a8a99627ce82fd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:46,113 DEBUG [StoreOpener-109e2ee6b532fbae23a8a99627ce82fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2 2023-02-09 14:55:46,113 DEBUG [StoreOpener-109e2ee6b532fbae23a8a99627ce82fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2 2023-02-09 14:55:46,113 INFO [StoreOpener-109e2ee6b532fbae23a8a99627ce82fd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 109e2ee6b532fbae23a8a99627ce82fd columnFamilyName cf2 2023-02-09 14:55:46,113 INFO [StoreOpener-109e2ee6b532fbae23a8a99627ce82fd-1] regionserver.HStore(310): Store=109e2ee6b532fbae23a8a99627ce82fd/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:46,114 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:46,114 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:46,115 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:55:46,116 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:46,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:55:46,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 109e2ee6b532fbae23a8a99627ce82fd; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72997456, jitterRate=0.08774685859680176}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:55:46,118 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 109e2ee6b532fbae23a8a99627ce82fd: Running coprocessor pre-open hook at 1675954546109Writing region info on filesystem at 1675954546109Initializing all the Stores at 1675954546109Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954546109Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954546109Cleaning up temporary data from old regions at 1675954546115 (+6 ms)Cleaning up detritus from prior splits at 1675954546115Running coprocessor post-open hooks at 1675954546118 (+3 ms)Region opened successfully at 1675954546118 2023-02-09 14:55:46,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd., pid=127, masterSystemTime=1675954546105 2023-02-09 14:55:46,120 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. 2023-02-09 14:55:46,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. 2023-02-09 14:55:46,121 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=109e2ee6b532fbae23a8a99627ce82fd, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:46,121 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675954546121"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954546121"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954546121"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954546121"}]},"ts":"1675954546121"} 2023-02-09 14:55:46,123 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-02-09 14:55:46,123 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 109e2ee6b532fbae23a8a99627ce82fd, server=jenkins-hbase4.apache.org,33423,1675954443133 in 168 msec 2023-02-09 14:55:46,125 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-02-09 14:55:46,125 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=109e2ee6b532fbae23a8a99627ce82fd, ASSIGN in 323 msec 2023-02-09 14:55:46,125 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 14:55:46,125 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954546125"}]},"ts":"1675954546125"} 2023-02-09 14:55:46,126 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-02-09 14:55:46,128 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 14:55:46,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 356 msec 2023-02-09 14:55:46,279 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:46,280 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:46,280 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:46,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,281 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-09 14:55:46,281 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:46,282 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-09 14:55:46,282 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-09 14:55:46,282 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,282 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,284 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:46,284 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:46,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:46,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:46,284 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:46,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:46,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:46,284 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:46,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:46,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:46,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,285 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:46,285 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,285 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:46,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,285 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:46,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,285 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,285 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:46,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,285 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:46,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,285 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,285 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,286 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 14:55:46,286 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,286 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:46,286 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 14:55:46,286 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:46,286 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,286 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 14:55:46,286 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 14:55:46,286 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:46,286 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 14:55:46,286 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 14:55:46,286 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:46,286 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:46,286 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 14:55:46,286 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 14:55:46,286 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 14:55:46,286 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. 2023-02-09 14:55:46,286 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 14:55:46,286 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. started... 2023-02-09 14:55:46,287 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 109e2ee6b532fbae23a8a99627ce82fd 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 14:55:46,288 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,288 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,289 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,289 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:46,289 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,289 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:46,296 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf1/17ce46f94e524b2e934c31e91a889269 2023-02-09 14:55:46,706 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf2/af4200d2d0bc434ba98b6bad729e4a1b 2023-02-09 14:55:46,710 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf1/17ce46f94e524b2e934c31e91a889269 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/17ce46f94e524b2e934c31e91a889269 2023-02-09 14:55:46,712 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/17ce46f94e524b2e934c31e91a889269, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 14:55:46,713 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf2/af4200d2d0bc434ba98b6bad729e4a1b as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/af4200d2d0bc434ba98b6bad729e4a1b 2023-02-09 14:55:46,716 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/af4200d2d0bc434ba98b6bad729e4a1b, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 14:55:46,716 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 109e2ee6b532fbae23a8a99627ce82fd in 429ms, sequenceid=6, compaction requested=false 2023-02-09 14:55:46,717 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-02-09 14:55:46,717 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 109e2ee6b532fbae23a8a99627ce82fd: 2023-02-09 14:55:46,717 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. 2023-02-09 14:55:46,717 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:46,717 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:46,717 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:46,717 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 14:55:46,717 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 14:55:46,718 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,718 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:46,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:46,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:46,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:46,719 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,719 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:46,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:46,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:46,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:46,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:46,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:46,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:46,720 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-09 14:55:46,721 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4b4fc155[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:46,721 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-09 14:55:46,721 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,722 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,722 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,722 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,722 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,722 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,722 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 14:55:46,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,722 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,722 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 14:55:46,722 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 14:55:46,722 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:46,722 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:46,722 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 14:55:46,722 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:55:46,722 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:55:46,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,722 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 14:55:46,722 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 14:55:46,722 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:55:46,724 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 14:55:46,724 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:46,724 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 14:55:46,725 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:46,725 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 14:55:46,725 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 14:55:46,725 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 14:55:46,725 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:46,725 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:46,725 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 14:55:46,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:46,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:46,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:46,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:46,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:46,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:46,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:46,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:46,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:46,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:46,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:46,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:46,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,33423,1675954443133': 2023-02-09 14:55:46,728 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-09 14:55:46,728 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-09 14:55:46,728 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:46,728 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:46,728 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,728 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:46,730 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,730 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,730 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,730 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:46,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:46,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:46,730 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,730 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:46,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,730 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:46,730 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:46,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:46,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:46,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:46,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:46,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:46,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:46,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:46,731 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:46,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,731 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:46,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:46,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:46,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:46,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:46,733 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:46,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:46,733 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:46,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,733 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:46,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:46,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:46,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:46,743 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:46,743 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:46,743 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:46,743 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:46,743 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:46,743 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:46,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:46,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:46,743 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:46,743 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:46,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:46,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:46,743 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-09 14:55:46,743 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:46,743 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:46,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:46,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:46,743 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:46,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:46,743 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:46,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:46,743 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 14:55:46,743 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:46,744 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:46,744 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:46,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:46,744 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:46,744 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:46,744 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:46,744 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,744 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,844 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:46,844 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:46,844 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:46,845 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:46,854 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:46,854 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:46,854 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:46,856 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:46,856 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,856 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-09 14:55:46,856 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:46,856 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-09 14:55:46,856 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-09 14:55:46,856 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,856 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,857 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:46,857 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:46,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:46,857 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:46,857 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:46,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:46,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:46,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:46,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:46,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:46,858 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:46,858 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:46,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:46,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,859 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:46,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:46,859 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,859 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 14:55:46,859 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,859 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,859 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 14:55:46,859 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 14:55:46,859 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,859 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:46,859 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 14:55:46,859 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:46,859 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 14:55:46,859 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 14:55:46,859 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 14:55:46,859 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 14:55:46,859 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 14:55:46,859 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,860 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 14:55:46,860 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:46,860 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 14:55:46,860 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 14:55:46,860 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 14:55:46,860 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. 2023-02-09 14:55:46,860 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:46,860 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:46,860 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. started... 2023-02-09 14:55:46,860 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 109e2ee6b532fbae23a8a99627ce82fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:55:46,861 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,861 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,861 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,861 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:46,861 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:46,862 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:47,245 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 14:55:47,268 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf1/f334b4fe58114b65bc82fa6b079d3340 2023-02-09 14:55:47,680 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf2/701d0450d86e41478a87d2c7e8635d0f 2023-02-09 14:55:47,683 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf1/f334b4fe58114b65bc82fa6b079d3340 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/f334b4fe58114b65bc82fa6b079d3340 2023-02-09 14:55:47,686 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/f334b4fe58114b65bc82fa6b079d3340, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 14:55:47,687 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf2/701d0450d86e41478a87d2c7e8635d0f as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/701d0450d86e41478a87d2c7e8635d0f 2023-02-09 14:55:47,690 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/701d0450d86e41478a87d2c7e8635d0f, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 14:55:47,691 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 109e2ee6b532fbae23a8a99627ce82fd in 831ms, sequenceid=19, compaction requested=false 2023-02-09 14:55:47,691 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 109e2ee6b532fbae23a8a99627ce82fd: 2023-02-09 14:55:47,691 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. 2023-02-09 14:55:47,691 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:47,691 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:47,691 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,691 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 14:55:47,691 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 14:55:47,694 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,694 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:47,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:47,694 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,694 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:47,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:47,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:47,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:47,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:47,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:47,695 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-09 14:55:47,696 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7cd17b69[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:47,696 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-09 14:55:47,696 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,697 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,697 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,697 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,697 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,697 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 14:55:47,697 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 14:55:47,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,697 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:55:47,697 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:47,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,697 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 14:55:47,697 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 14:55:47,697 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 14:55:47,697 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:55:47,697 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 14:55:47,697 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:55:47,698 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 14:55:47,698 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,698 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:47,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:47,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:47,698 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 14:55:47,699 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 14:55:47,699 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:47,699 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 14:55:47,701 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 14:55:47,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:47,701 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:47,701 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 14:55:47,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:47,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:47,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:47,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:47,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:47,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:47,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,33423,1675954443133': 2023-02-09 14:55:47,703 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-09 14:55:47,703 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-09 14:55:47,703 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:47,703 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:47,703 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,703 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:47,704 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,704 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:47,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:47,704 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:47,704 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,704 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,705 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:47,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,705 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:47,705 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:47,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:47,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:47,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:47,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:47,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:47,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:47,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:47,705 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,706 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:47,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:47,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:47,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:47,707 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:47,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:47,707 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,707 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:47,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:47,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:47,710 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:47,710 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:47,710 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:47,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:47,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:47,710 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:47,710 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,710 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-09 14:55:47,710 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:47,710 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:47,710 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:47,710 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,710 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:47,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:47,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:47,710 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 14:55:47,710 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:47,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:47,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:47,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:47,710 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:47,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:47,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:47,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:47,710 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,710 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:47,711 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:47,711 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:47,711 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:47,711 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:47,711 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,711 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,811 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:47,811 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:47,811 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:47,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:47,820 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:47,820 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:47,821 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:47,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:47,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,822 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-09 14:55:47,822 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:47,822 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-09 14:55:47,822 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-09 14:55:47,823 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,823 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,824 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:47,824 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:47,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:47,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:47,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:47,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:47,824 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:47,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:47,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:47,824 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,824 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,824 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:47,824 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,824 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,825 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:47,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:47,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:47,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,825 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,825 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,825 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,825 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 14:55:47,825 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,825 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 14:55:47,825 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 14:55:47,825 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 14:55:47,825 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:47,825 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,825 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 14:55:47,825 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 14:55:47,825 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,826 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:47,826 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 14:55:47,826 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 14:55:47,826 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 14:55:47,826 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 14:55:47,825 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 14:55:47,826 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:47,826 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 14:55:47,826 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 14:55:47,826 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:47,826 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:47,826 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. 2023-02-09 14:55:47,826 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. started... 2023-02-09 14:55:47,826 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 109e2ee6b532fbae23a8a99627ce82fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:55:47,828 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,828 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,828 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,829 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:47,829 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,829 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:47,836 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf1/9f2e3516efb64f778d5e4e973f55350f 2023-02-09 14:55:47,847 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf2/595b9bc730d7470e8da9c19eefa6848f 2023-02-09 14:55:47,850 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf1/9f2e3516efb64f778d5e4e973f55350f as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/9f2e3516efb64f778d5e4e973f55350f 2023-02-09 14:55:47,853 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/9f2e3516efb64f778d5e4e973f55350f, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 14:55:47,854 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf2/595b9bc730d7470e8da9c19eefa6848f as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/595b9bc730d7470e8da9c19eefa6848f 2023-02-09 14:55:47,856 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/595b9bc730d7470e8da9c19eefa6848f, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 14:55:47,857 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 109e2ee6b532fbae23a8a99627ce82fd in 31ms, sequenceid=32, compaction requested=true 2023-02-09 14:55:47,857 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 109e2ee6b532fbae23a8a99627ce82fd: 2023-02-09 14:55:47,857 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. 2023-02-09 14:55:47,857 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:47,857 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:47,857 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,857 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 14:55:47,857 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 14:55:47,859 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,859 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:47,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:47,859 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,859 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:47,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:47,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:47,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:47,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:47,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:47,861 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-09 14:55:47,861 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@147bdb8b[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:47,861 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-09 14:55:47,861 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,862 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,862 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,862 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,862 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 14:55:47,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,862 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 14:55:47,862 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:55:47,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,862 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,862 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:47,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,862 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 14:55:47,862 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 14:55:47,862 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 14:55:47,862 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 14:55:47,862 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:55:47,862 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:55:47,867 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,867 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 14:55:47,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:47,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:47,867 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:47,867 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 14:55:47,867 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 14:55:47,867 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 14:55:47,867 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:47,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:47,867 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 14:55:47,867 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:47,867 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 14:55:47,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:47,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:47,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:47,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:47,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:47,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:47,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,33423,1675954443133': 2023-02-09 14:55:47,870 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-09 14:55:47,870 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-09 14:55:47,870 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:47,870 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:47,870 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,870 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:47,871 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,871 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:47,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:47,871 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,871 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,871 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:47,871 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:47,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:47,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,871 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:47,872 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:47,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:47,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:47,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:47,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:47,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:47,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:47,872 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:47,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,872 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:47,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:47,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:47,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:47,874 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:47,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,874 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:47,874 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:47,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:47,876 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:47,876 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:47,876 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:47,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:47,877 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-09 14:55:47,877 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:47,876 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:47,876 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:47,876 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:47,877 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,877 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:47,877 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 14:55:47,877 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:47,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:47,877 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:47,877 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:47,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:47,877 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:47,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:47,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:47,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:47,877 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:47,877 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:47,878 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,878 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,878 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:47,878 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:47,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:47,878 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:47,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:47,977 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:47,977 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:47,977 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:47,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:47,987 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:47,987 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:47,987 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:47,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:47,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,989 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-09 14:55:47,989 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:47,989 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-09 14:55:47,989 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-09 14:55:47,989 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,989 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,991 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:47,991 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:47,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:47,991 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:47,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:47,991 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,991 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:47,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:47,992 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:47,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:47,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,992 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:47,992 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:47,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,992 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,992 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,992 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:47,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,992 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:47,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:47,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,993 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 14:55:47,993 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,993 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,993 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,993 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 14:55:47,993 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:47,993 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 14:55:47,993 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 14:55:47,993 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 14:55:47,993 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 14:55:47,993 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 14:55:47,993 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,993 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,993 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 14:55:47,993 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 14:55:47,993 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:47,993 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:47,993 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 14:55:47,993 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 14:55:47,993 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 14:55:47,993 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 14:55:47,993 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:47,994 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:47,994 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. 2023-02-09 14:55:47,994 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. started... 2023-02-09 14:55:47,994 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 109e2ee6b532fbae23a8a99627ce82fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:55:47,994 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,995 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,995 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,995 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:47,995 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:47,995 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:48,001 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf1/c767666c4705461b9c2d8a9fbfed0aa8 2023-02-09 14:55:48,011 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf2/0e0587587b864889aec67ab75196bb15 2023-02-09 14:55:48,014 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf1/c767666c4705461b9c2d8a9fbfed0aa8 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/c767666c4705461b9c2d8a9fbfed0aa8 2023-02-09 14:55:48,017 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/c767666c4705461b9c2d8a9fbfed0aa8, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 14:55:48,018 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf2/0e0587587b864889aec67ab75196bb15 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/0e0587587b864889aec67ab75196bb15 2023-02-09 14:55:48,021 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/0e0587587b864889aec67ab75196bb15, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 14:55:48,022 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 109e2ee6b532fbae23a8a99627ce82fd in 28ms, sequenceid=45, compaction requested=true 2023-02-09 14:55:48,022 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 109e2ee6b532fbae23a8a99627ce82fd: 2023-02-09 14:55:48,022 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. 2023-02-09 14:55:48,022 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:48,022 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:48,022 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,022 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 14:55:48,022 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 14:55:48,023 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,023 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:48,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:48,024 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:48,024 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:48,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:48,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:48,026 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-09 14:55:48,026 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@43cb86a[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:48,026 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-09 14:55:48,026 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,028 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,028 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,028 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,028 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,028 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 14:55:48,028 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 14:55:48,028 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:55:48,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,028 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 14:55:48,028 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 14:55:48,028 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 14:55:48,028 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:48,028 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:55:48,028 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 14:55:48,028 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:55:48,029 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 14:55:48,029 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:48,029 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 14:55:48,030 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,030 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 14:55:48,030 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 14:55:48,030 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:48,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:48,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:48,030 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 14:55:48,030 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:48,031 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 14:55:48,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:48,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:48,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:48,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,33423,1675954443133': 2023-02-09 14:55:48,034 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-09 14:55:48,034 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-09 14:55:48,034 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:48,034 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:48,034 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,034 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:48,036 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,036 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:48,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:48,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,036 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:48,036 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,036 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:48,036 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,036 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:48,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:48,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:48,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:48,037 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,037 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:48,038 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:48,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,039 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,040 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:48,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:48,040 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:48,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:48,040 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,046 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,046 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:48,046 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:48,046 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:48,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:48,046 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-09 14:55:48,046 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:48,046 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:48,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:48,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:48,046 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,046 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:48,046 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:48,046 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 14:55:48,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:48,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:48,046 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:48,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:48,046 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:48,047 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:48,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:48,046 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:48,047 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:48,047 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,047 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,047 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:48,047 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:48,047 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,047 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,147 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:48,147 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:48,147 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:48,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:48,157 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:48,158 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:48,158 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:48,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:48,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,159 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-09 14:55:48,159 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:48,159 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-09 14:55:48,159 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-09 14:55:48,160 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,160 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,161 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,161 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:48,161 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:48,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:48,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:48,161 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:48,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:48,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:48,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:48,161 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,161 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:48,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:48,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,162 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,162 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:48,162 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,162 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:48,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,162 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:48,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,162 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 14:55:48,162 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,163 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 14:55:48,163 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,163 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 14:55:48,163 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 14:55:48,162 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,163 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,163 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 14:55:48,163 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:48,163 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:48,163 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 14:55:48,163 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 14:55:48,163 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 14:55:48,163 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 14:55:48,163 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 14:55:48,163 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:48,163 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 14:55:48,163 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 14:55:48,163 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 14:55:48,163 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:48,163 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:48,163 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. 2023-02-09 14:55:48,163 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. started... 2023-02-09 14:55:48,163 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 109e2ee6b532fbae23a8a99627ce82fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:55:48,164 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,164 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,165 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,165 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:48,165 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,165 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:48,171 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf1/6cb7d6ee6da3479c86f3a7c515a5c385 2023-02-09 14:55:48,181 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf2/6a5d53fe6f444b7c90b8959ca3f58956 2023-02-09 14:55:48,184 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf1/6cb7d6ee6da3479c86f3a7c515a5c385 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/6cb7d6ee6da3479c86f3a7c515a5c385 2023-02-09 14:55:48,186 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/6cb7d6ee6da3479c86f3a7c515a5c385, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 14:55:48,187 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf2/6a5d53fe6f444b7c90b8959ca3f58956 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/6a5d53fe6f444b7c90b8959ca3f58956 2023-02-09 14:55:48,190 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/6a5d53fe6f444b7c90b8959ca3f58956, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 14:55:48,190 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 109e2ee6b532fbae23a8a99627ce82fd in 27ms, sequenceid=58, compaction requested=true 2023-02-09 14:55:48,190 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 109e2ee6b532fbae23a8a99627ce82fd: 2023-02-09 14:55:48,191 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. 2023-02-09 14:55:48,191 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:48,191 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:48,191 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,191 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 14:55:48,191 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 14:55:48,193 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,193 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:48,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:48,193 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,193 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:48,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:48,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:48,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:48,195 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-09 14:55:48,195 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@314c84d1[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:48,195 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-09 14:55:48,195 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,196 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,196 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,196 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,197 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 14:55:48,197 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 14:55:48,197 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 14:55:48,197 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 14:55:48,197 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,197 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:48,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,197 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:55:48,197 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:55:48,197 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 14:55:48,197 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 14:55:48,197 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:55:48,199 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 14:55:48,199 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 14:55:48,199 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:48,199 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:48,199 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 14:55:48,199 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 14:55:48,200 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 14:55:48,200 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,200 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:48,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:48,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:48,200 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 14:55:48,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:48,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:48,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:48,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,33423,1675954443133': 2023-02-09 14:55:48,203 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-09 14:55:48,203 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-09 14:55:48,203 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:48,203 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:48,203 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,203 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:48,209 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,209 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:48,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:48,209 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,209 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,210 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:48,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:48,210 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:48,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,210 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,210 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:48,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:48,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:48,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:48,210 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:48,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:48,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:48,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:48,211 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:48,212 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,213 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,213 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,216 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,216 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,216 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:48,216 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:48,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:48,216 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-09 14:55:48,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:48,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:48,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:48,216 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:48,216 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:48,216 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:48,216 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:48,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:48,216 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:48,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:48,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:48,217 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:48,217 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:48,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:48,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:48,217 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,217 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,217 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,217 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,217 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,217 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,217 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,217 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 14:55:48,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:48,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:48,217 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,218 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:48,318 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:48,318 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:48,318 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:48,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:48,328 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:48,329 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:48,329 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:48,331 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:48,331 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,331 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-09 14:55:48,331 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:48,331 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-09 14:55:48,331 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-09 14:55:48,332 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,332 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,333 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:48,333 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:48,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:48,333 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,333 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:48,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:48,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:48,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:48,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:48,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:48,333 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,334 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:48,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,334 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,334 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,334 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:48,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,334 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:48,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,335 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,335 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 14:55:48,335 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:48,335 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 14:55:48,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,335 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 14:55:48,335 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:48,335 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,335 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,335 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 14:55:48,335 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 14:55:48,335 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 14:55:48,335 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:48,335 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 14:55:48,335 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 14:55:48,335 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 14:55:48,335 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 14:55:48,335 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-09 14:55:48,335 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,335 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-09 14:55:48,335 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:48,335 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-09 14:55:48,335 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:48,336 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:48,336 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. 2023-02-09 14:55:48,336 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. started... 2023-02-09 14:55:48,336 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 109e2ee6b532fbae23a8a99627ce82fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:55:48,337 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,337 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,338 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,338 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,338 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:48,338 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:48,347 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf1/54d3308a08234d96a61522b5e62fdc06 2023-02-09 14:55:48,358 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf2/3ed8ebfa31c946a5b358bd4fc871d7e1 2023-02-09 14:55:48,361 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf1/54d3308a08234d96a61522b5e62fdc06 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/54d3308a08234d96a61522b5e62fdc06 2023-02-09 14:55:48,364 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/54d3308a08234d96a61522b5e62fdc06, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 14:55:48,365 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf2/3ed8ebfa31c946a5b358bd4fc871d7e1 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/3ed8ebfa31c946a5b358bd4fc871d7e1 2023-02-09 14:55:48,368 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/3ed8ebfa31c946a5b358bd4fc871d7e1, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 14:55:48,369 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 109e2ee6b532fbae23a8a99627ce82fd in 33ms, sequenceid=71, compaction requested=true 2023-02-09 14:55:48,369 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 109e2ee6b532fbae23a8a99627ce82fd: 2023-02-09 14:55:48,369 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. 2023-02-09 14:55:48,369 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:48,369 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:48,369 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,369 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-09 14:55:48,369 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-09 14:55:48,372 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,372 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:48,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:48,372 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,372 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:48,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:48,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:48,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:48,374 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-09 14:55:48,374 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@46b1aba2[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:48,374 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-09 14:55:48,374 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,377 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,377 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,377 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,377 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 14:55:48,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,377 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 14:55:48,377 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,377 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:48,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,377 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:55:48,378 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 14:55:48,378 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 14:55:48,378 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:55:48,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,378 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-09 14:55:48,378 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-09 14:55:48,378 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:55:48,379 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 14:55:48,379 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,379 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:48,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:48,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:48,379 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 14:55:48,379 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 14:55:48,379 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:48,379 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 14:55:48,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:48,380 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-09 14:55:48,380 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:48,380 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-09 14:55:48,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:48,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:48,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,33423,1675954443133': 2023-02-09 14:55:48,383 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-09 14:55:48,383 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-09 14:55:48,383 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:48,383 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:48,383 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,383 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:48,385 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,385 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,385 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:48,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:48,385 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,385 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:48,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,385 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:48,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,385 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,385 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:48,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:48,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:48,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:48,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:48,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:48,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:48,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:48,386 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,386 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:48,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:48,387 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,388 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,388 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,391 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,391 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,391 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:48,391 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:48,391 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,391 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-09 14:55:48,391 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:48,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:48,391 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:48,391 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:48,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:48,391 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:48,391 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:48,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:48,391 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-09 14:55:48,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:48,391 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:48,391 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,391 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:48,391 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:48,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:48,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:48,392 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:48,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:48,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:48,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:48,392 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,392 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:48,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:48,392 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:48,392 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,392 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-09 14:55:48,492 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:48,492 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:48,492 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:48,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:48,499 DEBUG [Listener at localhost/40823] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,500 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=109e2ee6b532fbae23a8a99627ce82fd, daughterA=29257a2a7fe73e90adb45bdafa6efa33, daughterB=c977f0f6238f5b099c4fb893ca8719f7 2023-02-09 14:55:48,501 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=109e2ee6b532fbae23a8a99627ce82fd, daughterA=29257a2a7fe73e90adb45bdafa6efa33, daughterB=c977f0f6238f5b099c4fb893ca8719f7 2023-02-09 14:55:48,501 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=109e2ee6b532fbae23a8a99627ce82fd, daughterA=29257a2a7fe73e90adb45bdafa6efa33, daughterB=c977f0f6238f5b099c4fb893ca8719f7 2023-02-09 14:55:48,501 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=109e2ee6b532fbae23a8a99627ce82fd, daughterA=29257a2a7fe73e90adb45bdafa6efa33, daughterB=c977f0f6238f5b099c4fb893ca8719f7 2023-02-09 14:55:48,502 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=109e2ee6b532fbae23a8a99627ce82fd, UNASSIGN}] 2023-02-09 14:55:48,503 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=109e2ee6b532fbae23a8a99627ce82fd, UNASSIGN 2023-02-09 14:55:48,504 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=109e2ee6b532fbae23a8a99627ce82fd, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,504 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675954548503"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954548503"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954548503"}]},"ts":"1675954548503"} 2023-02-09 14:55:48,505 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 109e2ee6b532fbae23a8a99627ce82fd, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:55:48,655 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,655 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 109e2ee6b532fbae23a8a99627ce82fd, disabling compactions & flushes 2023-02-09 14:55:48,655 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. 2023-02-09 14:55:48,655 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. 2023-02-09 14:55:48,655 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. after waiting 0 ms 2023-02-09 14:55:48,655 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. 2023-02-09 14:55:48,655 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 109e2ee6b532fbae23a8a99627ce82fd 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 14:55:48,665 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf1/92e68d7b3a9649c8afcad149e9bf1c73 2023-02-09 14:55:48,678 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf2/e626cbb12e0049a4bcc2cf34830787cd 2023-02-09 14:55:48,682 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf1/92e68d7b3a9649c8afcad149e9bf1c73 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/92e68d7b3a9649c8afcad149e9bf1c73 2023-02-09 14:55:48,685 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/92e68d7b3a9649c8afcad149e9bf1c73, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 14:55:48,685 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.tmp/cf2/e626cbb12e0049a4bcc2cf34830787cd as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/e626cbb12e0049a4bcc2cf34830787cd 2023-02-09 14:55:48,689 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/e626cbb12e0049a4bcc2cf34830787cd, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 14:55:48,689 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 109e2ee6b532fbae23a8a99627ce82fd in 34ms, sequenceid=82, compaction requested=true 2023-02-09 14:55:48,693 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 14:55:48,694 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:55:48,694 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. 2023-02-09 14:55:48,695 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 109e2ee6b532fbae23a8a99627ce82fd: Waiting for close lock at 1675954548655Running coprocessor pre-close hooks at 1675954548655Disabling compacts and flushes for region at 1675954548655Disabling writes for close at 1675954548655Obtaining lock to block concurrent updates at 1675954548655Preparing flush snapshotting stores in 109e2ee6b532fbae23a8a99627ce82fd at 1675954548655Finished memstore snapshotting testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675954548656 (+1 ms)Flushing stores of testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. at 1675954548656Flushing 109e2ee6b532fbae23a8a99627ce82fd/cf1: creating writer at 1675954548658 (+2 ms)Flushing 109e2ee6b532fbae23a8a99627ce82fd/cf1: appending metadata at 1675954548660 (+2 ms)Flushing 109e2ee6b532fbae23a8a99627ce82fd/cf1: closing flushed file at 1675954548660Flushing 109e2ee6b532fbae23a8a99627ce82fd/cf2: creating writer at 1675954548669 (+9 ms)Flushing 109e2ee6b532fbae23a8a99627ce82fd/cf2: appending metadata at 1675954548670 (+1 ms)Flushing 109e2ee6b532fbae23a8a99627ce82fd/cf2: closing flushed file at 1675954548670Flushing 109e2ee6b532fbae23a8a99627ce82fd/cf1: reopening flushed file at 1675954548682 (+12 ms)Flushing 109e2ee6b532fbae23a8a99627ce82fd/cf2: reopening flushed file at 1675954548686 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 109e2ee6b532fbae23a8a99627ce82fd in 34ms, sequenceid=82, compaction requested=true at 1675954548689 (+3 ms)Writing region close event to WAL at 1675954548691 (+2 ms)Running coprocessor post-close hooks at 1675954548694 (+3 ms)Closed at 1675954548694 2023-02-09 14:55:48,696 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,696 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=109e2ee6b532fbae23a8a99627ce82fd, regionState=CLOSED 2023-02-09 14:55:48,696 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675954548696"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954548696"}]},"ts":"1675954548696"} 2023-02-09 14:55:48,698 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-02-09 14:55:48,698 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 109e2ee6b532fbae23a8a99627ce82fd, server=jenkins-hbase4.apache.org,33423,1675954443133 in 192 msec 2023-02-09 14:55:48,700 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-02-09 14:55:48,700 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=109e2ee6b532fbae23a8a99627ce82fd, UNASSIGN in 197 msec 2023-02-09 14:55:48,708 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=109e2ee6b532fbae23a8a99627ce82fd, threads=14 2023-02-09 14:55:48,709 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/0e0587587b864889aec67ab75196bb15 for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,709 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/3ed8ebfa31c946a5b358bd4fc871d7e1 for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,709 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/595b9bc730d7470e8da9c19eefa6848f for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,709 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/6a5d53fe6f444b7c90b8959ca3f58956 for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,709 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/701d0450d86e41478a87d2c7e8635d0f for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,709 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/af4200d2d0bc434ba98b6bad729e4a1b for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,709 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/e626cbb12e0049a4bcc2cf34830787cd for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,709 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/17ce46f94e524b2e934c31e91a889269 for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,709 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/6cb7d6ee6da3479c86f3a7c515a5c385 for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,709 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/92e68d7b3a9649c8afcad149e9bf1c73 for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,710 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/9f2e3516efb64f778d5e4e973f55350f for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,710 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/54d3308a08234d96a61522b5e62fdc06 for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,710 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/c767666c4705461b9c2d8a9fbfed0aa8 for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,710 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/f334b4fe58114b65bc82fa6b079d3340 for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,712 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/3ed8ebfa31c946a5b358bd4fc871d7e1, top=false 2023-02-09 14:55:48,714 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/6a5d53fe6f444b7c90b8959ca3f58956, top=false 2023-02-09 14:55:48,714 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/595b9bc730d7470e8da9c19eefa6848f, top=false 2023-02-09 14:55:48,714 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/17ce46f94e524b2e934c31e91a889269, top=false 2023-02-09 14:55:48,715 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/e626cbb12e0049a4bcc2cf34830787cd, top=false 2023-02-09 14:55:48,715 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/92e68d7b3a9649c8afcad149e9bf1c73, top=false 2023-02-09 14:55:48,718 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.splits/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-6a5d53fe6f444b7c90b8959ca3f58956 for child: 29257a2a7fe73e90adb45bdafa6efa33, parent: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,720 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/6a5d53fe6f444b7c90b8959ca3f58956 for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,721 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/701d0450d86e41478a87d2c7e8635d0f, top=false 2023-02-09 14:55:48,722 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/af4200d2d0bc434ba98b6bad729e4a1b, top=false 2023-02-09 14:55:48,722 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/54d3308a08234d96a61522b5e62fdc06, top=false 2023-02-09 14:55:48,722 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/0e0587587b864889aec67ab75196bb15, top=false 2023-02-09 14:55:48,724 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/9f2e3516efb64f778d5e4e973f55350f, top=false 2023-02-09 14:55:48,725 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/c767666c4705461b9c2d8a9fbfed0aa8, top=false 2023-02-09 14:55:48,727 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/6cb7d6ee6da3479c86f3a7c515a5c385, top=false 2023-02-09 14:55:48,730 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/f334b4fe58114b65bc82fa6b079d3340, top=false 2023-02-09 14:55:48,748 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.splits/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-92e68d7b3a9649c8afcad149e9bf1c73 for child: 29257a2a7fe73e90adb45bdafa6efa33, parent: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,751 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.splits/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-af4200d2d0bc434ba98b6bad729e4a1b for child: 29257a2a7fe73e90adb45bdafa6efa33, parent: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,751 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.splits/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-f334b4fe58114b65bc82fa6b079d3340 for child: 29257a2a7fe73e90adb45bdafa6efa33, parent: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,752 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.splits/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-9f2e3516efb64f778d5e4e973f55350f for child: 29257a2a7fe73e90adb45bdafa6efa33, parent: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,752 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.splits/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-c767666c4705461b9c2d8a9fbfed0aa8 for child: 29257a2a7fe73e90adb45bdafa6efa33, parent: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,752 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/92e68d7b3a9649c8afcad149e9bf1c73 for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,754 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/f334b4fe58114b65bc82fa6b079d3340 for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,755 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/af4200d2d0bc434ba98b6bad729e4a1b for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,756 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/9f2e3516efb64f778d5e4e973f55350f for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,756 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/c767666c4705461b9c2d8a9fbfed0aa8 for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,756 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.splits/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-595b9bc730d7470e8da9c19eefa6848f for child: 29257a2a7fe73e90adb45bdafa6efa33, parent: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,756 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.splits/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-54d3308a08234d96a61522b5e62fdc06 for child: 29257a2a7fe73e90adb45bdafa6efa33, parent: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,756 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.splits/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-e626cbb12e0049a4bcc2cf34830787cd for child: 29257a2a7fe73e90adb45bdafa6efa33, parent: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,757 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.splits/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-701d0450d86e41478a87d2c7e8635d0f for child: 29257a2a7fe73e90adb45bdafa6efa33, parent: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,757 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.splits/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-6cb7d6ee6da3479c86f3a7c515a5c385 for child: 29257a2a7fe73e90adb45bdafa6efa33, parent: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,759 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/595b9bc730d7470e8da9c19eefa6848f for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,760 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/54d3308a08234d96a61522b5e62fdc06 for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,761 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/6cb7d6ee6da3479c86f3a7c515a5c385 for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,761 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/701d0450d86e41478a87d2c7e8635d0f for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,761 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.splits/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-3ed8ebfa31c946a5b358bd4fc871d7e1 for child: 29257a2a7fe73e90adb45bdafa6efa33, parent: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,762 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/e626cbb12e0049a4bcc2cf34830787cd for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,762 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.splits/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-17ce46f94e524b2e934c31e91a889269 for child: 29257a2a7fe73e90adb45bdafa6efa33, parent: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,764 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/.splits/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-0e0587587b864889aec67ab75196bb15 for child: 29257a2a7fe73e90adb45bdafa6efa33, parent: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,764 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/3ed8ebfa31c946a5b358bd4fc871d7e1 for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,765 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/17ce46f94e524b2e934c31e91a889269 for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,767 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/0e0587587b864889aec67ab75196bb15 for region: 109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:48,767 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 109e2ee6b532fbae23a8a99627ce82fd Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-02-09 14:55:48,789 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 14:55:48,790 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/c977f0f6238f5b099c4fb893ca8719f7/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 14:55:48,791 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1675954548791"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1675954548791"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1675954548791"}]},"ts":"1675954548791"} 2023-02-09 14:55:48,791 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675954548791"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954548791"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954548791"}]},"ts":"1675954548791"} 2023-02-09 14:55:48,791 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675954548499.c977f0f6238f5b099c4fb893ca8719f7.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675954548791"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954548791"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954548791"}]},"ts":"1675954548791"} 2023-02-09 14:55:48,794 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=29257a2a7fe73e90adb45bdafa6efa33, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c977f0f6238f5b099c4fb893ca8719f7, ASSIGN}] 2023-02-09 14:55:48,795 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c977f0f6238f5b099c4fb893ca8719f7, ASSIGN 2023-02-09 14:55:48,795 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c977f0f6238f5b099c4fb893ca8719f7, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33423,1675954443133; forceNewPlan=false, retain=false 2023-02-09 14:55:48,796 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=29257a2a7fe73e90adb45bdafa6efa33, ASSIGN 2023-02-09 14:55:48,796 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=29257a2a7fe73e90adb45bdafa6efa33, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33423,1675954443133; forceNewPlan=false, retain=false 2023-02-09 14:55:48,945 INFO [jenkins-hbase4:43829] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 14:55:48,947 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=c977f0f6238f5b099c4fb893ca8719f7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,947 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675954548499.c977f0f6238f5b099c4fb893ca8719f7.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675954548947"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954548947"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954548947"}]},"ts":"1675954548947"} 2023-02-09 14:55:48,948 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=132, state=RUNNABLE; OpenRegionProcedure c977f0f6238f5b099c4fb893ca8719f7, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:55:48,950 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=29257a2a7fe73e90adb45bdafa6efa33, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:48,950 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675954548950"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954548950"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954548950"}]},"ts":"1675954548950"} 2023-02-09 14:55:48,951 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=131, state=RUNNABLE; OpenRegionProcedure 29257a2a7fe73e90adb45bdafa6efa33, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:55:49,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,71,1675954548499.c977f0f6238f5b099c4fb893ca8719f7. 2023-02-09 14:55:49,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c977f0f6238f5b099c4fb893ca8719f7, NAME => 'testSplitTableRegionEmptyDaughter,71,1675954548499.c977f0f6238f5b099c4fb893ca8719f7.', STARTKEY => '71', ENDKEY => ''} 2023-02-09 14:55:49,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:55:49,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter c977f0f6238f5b099c4fb893ca8719f7 2023-02-09 14:55:49,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,71,1675954548499.c977f0f6238f5b099c4fb893ca8719f7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:49,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for c977f0f6238f5b099c4fb893ca8719f7 2023-02-09 14:55:49,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for c977f0f6238f5b099c4fb893ca8719f7 2023-02-09 14:55:49,104 INFO [StoreOpener-c977f0f6238f5b099c4fb893ca8719f7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c977f0f6238f5b099c4fb893ca8719f7 2023-02-09 14:55:49,105 DEBUG [StoreOpener-c977f0f6238f5b099c4fb893ca8719f7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/c977f0f6238f5b099c4fb893ca8719f7/cf1 2023-02-09 14:55:49,105 DEBUG [StoreOpener-c977f0f6238f5b099c4fb893ca8719f7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/c977f0f6238f5b099c4fb893ca8719f7/cf1 2023-02-09 14:55:49,106 INFO [StoreOpener-c977f0f6238f5b099c4fb893ca8719f7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c977f0f6238f5b099c4fb893ca8719f7 columnFamilyName cf1 2023-02-09 14:55:49,106 INFO [StoreOpener-c977f0f6238f5b099c4fb893ca8719f7-1] regionserver.HStore(310): Store=c977f0f6238f5b099c4fb893ca8719f7/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:49,106 INFO [StoreOpener-c977f0f6238f5b099c4fb893ca8719f7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c977f0f6238f5b099c4fb893ca8719f7 2023-02-09 14:55:49,107 DEBUG [StoreOpener-c977f0f6238f5b099c4fb893ca8719f7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/c977f0f6238f5b099c4fb893ca8719f7/cf2 2023-02-09 14:55:49,107 DEBUG [StoreOpener-c977f0f6238f5b099c4fb893ca8719f7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/c977f0f6238f5b099c4fb893ca8719f7/cf2 2023-02-09 14:55:49,107 INFO [StoreOpener-c977f0f6238f5b099c4fb893ca8719f7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c977f0f6238f5b099c4fb893ca8719f7 columnFamilyName cf2 2023-02-09 14:55:49,108 INFO [StoreOpener-c977f0f6238f5b099c4fb893ca8719f7-1] regionserver.HStore(310): Store=c977f0f6238f5b099c4fb893ca8719f7/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:49,108 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/c977f0f6238f5b099c4fb893ca8719f7 2023-02-09 14:55:49,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/c977f0f6238f5b099c4fb893ca8719f7 2023-02-09 14:55:49,110 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:55:49,111 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for c977f0f6238f5b099c4fb893ca8719f7 2023-02-09 14:55:49,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened c977f0f6238f5b099c4fb893ca8719f7; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64159954, jitterRate=-0.04394218325614929}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:55:49,111 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for c977f0f6238f5b099c4fb893ca8719f7: Running coprocessor pre-open hook at 1675954549103Writing region info on filesystem at 1675954549103Initializing all the Stores at 1675954549104 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954549104Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954549104Cleaning up temporary data from old regions at 1675954549109 (+5 ms)Cleaning up detritus from prior splits at 1675954549110 (+1 ms)Running coprocessor post-open hooks at 1675954549111 (+1 ms)Region opened successfully at 1675954549111 2023-02-09 14:55:49,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1675954548499.c977f0f6238f5b099c4fb893ca8719f7., pid=133, masterSystemTime=1675954549099 2023-02-09 14:55:49,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1675954548499.c977f0f6238f5b099c4fb893ca8719f7. 2023-02-09 14:55:49,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,71,1675954548499.c977f0f6238f5b099c4fb893ca8719f7. 2023-02-09 14:55:49,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33. 2023-02-09 14:55:49,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 29257a2a7fe73e90adb45bdafa6efa33, NAME => 'testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.', STARTKEY => '', ENDKEY => '71'} 2023-02-09 14:55:49,114 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=c977f0f6238f5b099c4fb893ca8719f7, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:49,114 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:55:49,114 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1675954548499.c977f0f6238f5b099c4fb893ca8719f7.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675954549114"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954549114"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954549114"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954549114"}]},"ts":"1675954549114"} 2023-02-09 14:55:49,114 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 29257a2a7fe73e90adb45bdafa6efa33 2023-02-09 14:55:49,114 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:49,114 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 29257a2a7fe73e90adb45bdafa6efa33 2023-02-09 14:55:49,114 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 29257a2a7fe73e90adb45bdafa6efa33 2023-02-09 14:55:49,115 INFO [StoreOpener-29257a2a7fe73e90adb45bdafa6efa33-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 29257a2a7fe73e90adb45bdafa6efa33 2023-02-09 14:55:49,116 DEBUG [StoreOpener-29257a2a7fe73e90adb45bdafa6efa33-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1 2023-02-09 14:55:49,116 DEBUG [StoreOpener-29257a2a7fe73e90adb45bdafa6efa33-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1 2023-02-09 14:55:49,116 INFO [StoreOpener-29257a2a7fe73e90adb45bdafa6efa33-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 29257a2a7fe73e90adb45bdafa6efa33 columnFamilyName cf1 2023-02-09 14:55:49,117 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=132 2023-02-09 14:55:49,117 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=132, state=SUCCESS; OpenRegionProcedure c977f0f6238f5b099c4fb893ca8719f7, server=jenkins-hbase4.apache.org,33423,1675954443133 in 167 msec 2023-02-09 14:55:49,118 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c977f0f6238f5b099c4fb893ca8719f7, ASSIGN in 323 msec 2023-02-09 14:55:49,121 DEBUG [StoreOpener-29257a2a7fe73e90adb45bdafa6efa33-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-17ce46f94e524b2e934c31e91a889269 2023-02-09 14:55:49,124 DEBUG [StoreOpener-29257a2a7fe73e90adb45bdafa6efa33-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-54d3308a08234d96a61522b5e62fdc06 2023-02-09 14:55:49,127 DEBUG [StoreOpener-29257a2a7fe73e90adb45bdafa6efa33-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-6cb7d6ee6da3479c86f3a7c515a5c385 2023-02-09 14:55:49,129 DEBUG [StoreOpener-29257a2a7fe73e90adb45bdafa6efa33-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-92e68d7b3a9649c8afcad149e9bf1c73 2023-02-09 14:55:49,132 DEBUG [StoreOpener-29257a2a7fe73e90adb45bdafa6efa33-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-9f2e3516efb64f778d5e4e973f55350f 2023-02-09 14:55:49,134 DEBUG [StoreOpener-29257a2a7fe73e90adb45bdafa6efa33-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-c767666c4705461b9c2d8a9fbfed0aa8 2023-02-09 14:55:49,136 DEBUG [StoreOpener-29257a2a7fe73e90adb45bdafa6efa33-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-f334b4fe58114b65bc82fa6b079d3340 2023-02-09 14:55:49,137 INFO [StoreOpener-29257a2a7fe73e90adb45bdafa6efa33-1] regionserver.HStore(310): Store=29257a2a7fe73e90adb45bdafa6efa33/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:49,137 INFO [StoreOpener-29257a2a7fe73e90adb45bdafa6efa33-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 29257a2a7fe73e90adb45bdafa6efa33 2023-02-09 14:55:49,137 DEBUG [StoreOpener-29257a2a7fe73e90adb45bdafa6efa33-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2 2023-02-09 14:55:49,137 DEBUG [StoreOpener-29257a2a7fe73e90adb45bdafa6efa33-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2 2023-02-09 14:55:49,138 INFO [StoreOpener-29257a2a7fe73e90adb45bdafa6efa33-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 29257a2a7fe73e90adb45bdafa6efa33 columnFamilyName cf2 2023-02-09 14:55:49,142 DEBUG [StoreOpener-29257a2a7fe73e90adb45bdafa6efa33-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-0e0587587b864889aec67ab75196bb15 2023-02-09 14:55:49,144 DEBUG [StoreOpener-29257a2a7fe73e90adb45bdafa6efa33-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-3ed8ebfa31c946a5b358bd4fc871d7e1 2023-02-09 14:55:49,147 DEBUG [StoreOpener-29257a2a7fe73e90adb45bdafa6efa33-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-595b9bc730d7470e8da9c19eefa6848f 2023-02-09 14:55:49,149 DEBUG [StoreOpener-29257a2a7fe73e90adb45bdafa6efa33-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-6a5d53fe6f444b7c90b8959ca3f58956 2023-02-09 14:55:49,151 DEBUG [StoreOpener-29257a2a7fe73e90adb45bdafa6efa33-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-701d0450d86e41478a87d2c7e8635d0f 2023-02-09 14:55:49,153 DEBUG [StoreOpener-29257a2a7fe73e90adb45bdafa6efa33-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-af4200d2d0bc434ba98b6bad729e4a1b 2023-02-09 14:55:49,156 DEBUG [StoreOpener-29257a2a7fe73e90adb45bdafa6efa33-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-e626cbb12e0049a4bcc2cf34830787cd 2023-02-09 14:55:49,156 INFO [StoreOpener-29257a2a7fe73e90adb45bdafa6efa33-1] regionserver.HStore(310): Store=29257a2a7fe73e90adb45bdafa6efa33/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:49,157 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33 2023-02-09 14:55:49,157 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33 2023-02-09 14:55:49,159 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:55:49,159 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 29257a2a7fe73e90adb45bdafa6efa33 2023-02-09 14:55:49,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 29257a2a7fe73e90adb45bdafa6efa33; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68284699, jitterRate=0.01752130687236786}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:55:49,160 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 29257a2a7fe73e90adb45bdafa6efa33: Running coprocessor pre-open hook at 1675954549114Writing region info on filesystem at 1675954549114Initializing all the Stores at 1675954549115 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954549115Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954549115Cleaning up temporary data from old regions at 1675954549158 (+43 ms)Cleaning up detritus from prior splits at 1675954549158Running coprocessor post-open hooks at 1675954549160 (+2 ms)Region opened successfully at 1675954549160 2023-02-09 14:55:49,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33., pid=134, masterSystemTime=1675954549099 2023-02-09 14:55:49,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:55:49,162 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 14:55:49,162 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 14:55:49,162 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 14:55:49,162 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-09 14:55:49,162 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-09 14:55:49,162 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HStore(1912): 29257a2a7fe73e90adb45bdafa6efa33/cf1 is initiating minor compaction (all files) 2023-02-09 14:55:49,162 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HStore(1912): 29257a2a7fe73e90adb45bdafa6efa33/cf2 is initiating minor compaction (all files) 2023-02-09 14:55:49,162 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 29257a2a7fe73e90adb45bdafa6efa33/cf1 in testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33. 2023-02-09 14:55:49,162 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 29257a2a7fe73e90adb45bdafa6efa33/cf2 in testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33. 2023-02-09 14:55:49,162 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-17ce46f94e524b2e934c31e91a889269, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-f334b4fe58114b65bc82fa6b079d3340, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-9f2e3516efb64f778d5e4e973f55350f, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-c767666c4705461b9c2d8a9fbfed0aa8, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-6cb7d6ee6da3479c86f3a7c515a5c385, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-54d3308a08234d96a61522b5e62fdc06, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-92e68d7b3a9649c8afcad149e9bf1c73] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/.tmp, totalSize=34.9 K 2023-02-09 14:55:49,162 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-af4200d2d0bc434ba98b6bad729e4a1b, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-701d0450d86e41478a87d2c7e8635d0f, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-595b9bc730d7470e8da9c19eefa6848f, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-0e0587587b864889aec67ab75196bb15, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-6a5d53fe6f444b7c90b8959ca3f58956, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-3ed8ebfa31c946a5b358bd4fc871d7e1, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-e626cbb12e0049a4bcc2cf34830787cd] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/.tmp, totalSize=34.9 K 2023-02-09 14:55:49,163 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-17ce46f94e524b2e934c31e91a889269, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675954546276 2023-02-09 14:55:49,163 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-af4200d2d0bc434ba98b6bad729e4a1b, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675954546276 2023-02-09 14:55:49,163 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-f334b4fe58114b65bc82fa6b079d3340, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675954546845 2023-02-09 14:55:49,163 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-701d0450d86e41478a87d2c7e8635d0f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675954546845 2023-02-09 14:55:49,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33. 2023-02-09 14:55:49,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33. 2023-02-09 14:55:49,164 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-9f2e3516efb64f778d5e4e973f55350f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675954547812 2023-02-09 14:55:49,164 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-595b9bc730d7470e8da9c19eefa6848f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675954547812 2023-02-09 14:55:49,164 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=29257a2a7fe73e90adb45bdafa6efa33, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:49,164 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-c767666c4705461b9c2d8a9fbfed0aa8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675954547979 2023-02-09 14:55:49,164 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675954549164"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954549164"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954549164"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954549164"}]},"ts":"1675954549164"} 2023-02-09 14:55:49,164 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-0e0587587b864889aec67ab75196bb15, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675954547979 2023-02-09 14:55:49,165 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-6cb7d6ee6da3479c86f3a7c515a5c385, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675954548149 2023-02-09 14:55:49,165 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-6a5d53fe6f444b7c90b8959ca3f58956, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675954548149 2023-02-09 14:55:49,165 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-54d3308a08234d96a61522b5e62fdc06, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675954548320 2023-02-09 14:55:49,165 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-3ed8ebfa31c946a5b358bd4fc871d7e1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675954548320 2023-02-09 14:55:49,165 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-92e68d7b3a9649c8afcad149e9bf1c73, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675954548493 2023-02-09 14:55:49,166 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-e626cbb12e0049a4bcc2cf34830787cd, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675954548493 2023-02-09 14:55:49,168 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=131 2023-02-09 14:55:49,168 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=131, state=SUCCESS; OpenRegionProcedure 29257a2a7fe73e90adb45bdafa6efa33, server=jenkins-hbase4.apache.org,33423,1675954443133 in 214 msec 2023-02-09 14:55:49,170 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=131, resume processing ppid=128 2023-02-09 14:55:49,170 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=29257a2a7fe73e90adb45bdafa6efa33, ASSIGN in 375 msec 2023-02-09 14:55:49,172 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=109e2ee6b532fbae23a8a99627ce82fd, daughterA=29257a2a7fe73e90adb45bdafa6efa33, daughterB=c977f0f6238f5b099c4fb893ca8719f7 in 672 msec 2023-02-09 14:55:49,184 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] throttle.PressureAwareThroughputController(145): 29257a2a7fe73e90adb45bdafa6efa33#cf1#compaction#126 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:55:49,184 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] throttle.PressureAwareThroughputController(145): 29257a2a7fe73e90adb45bdafa6efa33#cf2#compaction#127 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:55:49,197 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/.tmp/cf1/153e931814f34883bf32d3fb032365a7 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/153e931814f34883bf32d3fb032365a7 2023-02-09 14:55:49,197 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/.tmp/cf2/9d33547d42b54f7ab70889de30e5b1e2 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/9d33547d42b54f7ab70889de30e5b1e2 2023-02-09 14:55:49,201 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 29257a2a7fe73e90adb45bdafa6efa33/cf1 of 29257a2a7fe73e90adb45bdafa6efa33 into 153e931814f34883bf32d3fb032365a7(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:55:49,201 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 29257a2a7fe73e90adb45bdafa6efa33/cf2 of 29257a2a7fe73e90adb45bdafa6efa33 into 9d33547d42b54f7ab70889de30e5b1e2(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:55:49,201 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 29257a2a7fe73e90adb45bdafa6efa33: 2023-02-09 14:55:49,201 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 29257a2a7fe73e90adb45bdafa6efa33: 2023-02-09 14:55:49,201 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33., storeName=29257a2a7fe73e90adb45bdafa6efa33/cf1, priority=9, startTime=1675954549161; duration=0sec 2023-02-09 14:55:49,201 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33., storeName=29257a2a7fe73e90adb45bdafa6efa33/cf2, priority=9, startTime=1675954549162; duration=0sec 2023-02-09 14:55:49,201 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:55:49,202 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:55:49,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33423] ipc.CallRunner(144): callId: 987 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:47026 deadline: 1675954609250, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd. is not online on jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:49,359 DEBUG [Listener at localhost/40823] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-02-09 14:55:49,363 WARN [Listener at localhost/40823] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 14:55:49,363 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:49,363 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:49,364 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:49,364 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:49,364 INFO [Listener at localhost/40823] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-02-09 14:55:49,364 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegionEmptyDaughter 2023-02-09 14:55:49,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 14:55:49,366 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:49,366 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:49,367 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-09 14:55:49,367 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954549367"}]},"ts":"1675954549367"} 2023-02-09 14:55:49,368 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-02-09 14:55:49,369 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-02-09 14:55:49,370 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=29257a2a7fe73e90adb45bdafa6efa33, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c977f0f6238f5b099c4fb893ca8719f7, UNASSIGN}] 2023-02-09 14:55:49,370 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c977f0f6238f5b099c4fb893ca8719f7, UNASSIGN 2023-02-09 14:55:49,371 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=c977f0f6238f5b099c4fb893ca8719f7, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:49,371 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1675954548499.c977f0f6238f5b099c4fb893ca8719f7.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675954549371"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954549371"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954549371"}]},"ts":"1675954549371"} 2023-02-09 14:55:49,372 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure c977f0f6238f5b099c4fb893ca8719f7, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:55:49,373 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=29257a2a7fe73e90adb45bdafa6efa33, UNASSIGN 2023-02-09 14:55:49,374 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=29257a2a7fe73e90adb45bdafa6efa33, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:49,374 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675954549374"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954549374"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954549374"}]},"ts":"1675954549374"} 2023-02-09 14:55:49,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure 29257a2a7fe73e90adb45bdafa6efa33, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:55:49,467 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:49,467 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:49,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-09 14:55:49,523 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 29257a2a7fe73e90adb45bdafa6efa33 2023-02-09 14:55:49,523 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 29257a2a7fe73e90adb45bdafa6efa33, disabling compactions & flushes 2023-02-09 14:55:49,523 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33. 2023-02-09 14:55:49,523 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33. 2023-02-09 14:55:49,523 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33. after waiting 0 ms 2023-02-09 14:55:49,523 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33. 2023-02-09 14:55:49,524 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-17ce46f94e524b2e934c31e91a889269, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-f334b4fe58114b65bc82fa6b079d3340, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-9f2e3516efb64f778d5e4e973f55350f, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-c767666c4705461b9c2d8a9fbfed0aa8, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-6cb7d6ee6da3479c86f3a7c515a5c385, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-54d3308a08234d96a61522b5e62fdc06, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-92e68d7b3a9649c8afcad149e9bf1c73] to archive 2023-02-09 14:55:49,525 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:49,526 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-17ce46f94e524b2e934c31e91a889269 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-17ce46f94e524b2e934c31e91a889269 2023-02-09 14:55:49,527 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-f334b4fe58114b65bc82fa6b079d3340 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-f334b4fe58114b65bc82fa6b079d3340 2023-02-09 14:55:49,528 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-9f2e3516efb64f778d5e4e973f55350f to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-9f2e3516efb64f778d5e4e973f55350f 2023-02-09 14:55:49,529 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-c767666c4705461b9c2d8a9fbfed0aa8 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-c767666c4705461b9c2d8a9fbfed0aa8 2023-02-09 14:55:49,530 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-6cb7d6ee6da3479c86f3a7c515a5c385 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-6cb7d6ee6da3479c86f3a7c515a5c385 2023-02-09 14:55:49,531 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-54d3308a08234d96a61522b5e62fdc06 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-54d3308a08234d96a61522b5e62fdc06 2023-02-09 14:55:49,531 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-92e68d7b3a9649c8afcad149e9bf1c73 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-92e68d7b3a9649c8afcad149e9bf1c73 2023-02-09 14:55:49,533 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-af4200d2d0bc434ba98b6bad729e4a1b, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-701d0450d86e41478a87d2c7e8635d0f, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-595b9bc730d7470e8da9c19eefa6848f, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-0e0587587b864889aec67ab75196bb15, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-6a5d53fe6f444b7c90b8959ca3f58956, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-3ed8ebfa31c946a5b358bd4fc871d7e1, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-e626cbb12e0049a4bcc2cf34830787cd] to archive 2023-02-09 14:55:49,533 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:49,534 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-af4200d2d0bc434ba98b6bad729e4a1b to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-af4200d2d0bc434ba98b6bad729e4a1b 2023-02-09 14:55:49,535 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-701d0450d86e41478a87d2c7e8635d0f to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-701d0450d86e41478a87d2c7e8635d0f 2023-02-09 14:55:49,536 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-595b9bc730d7470e8da9c19eefa6848f to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-595b9bc730d7470e8da9c19eefa6848f 2023-02-09 14:55:49,537 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-0e0587587b864889aec67ab75196bb15 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-0e0587587b864889aec67ab75196bb15 2023-02-09 14:55:49,538 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-6a5d53fe6f444b7c90b8959ca3f58956 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-6a5d53fe6f444b7c90b8959ca3f58956 2023-02-09 14:55:49,539 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-3ed8ebfa31c946a5b358bd4fc871d7e1 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-3ed8ebfa31c946a5b358bd4fc871d7e1 2023-02-09 14:55:49,540 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-e626cbb12e0049a4bcc2cf34830787cd to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/testSplitTableRegionEmptyDaughter=109e2ee6b532fbae23a8a99627ce82fd-e626cbb12e0049a4bcc2cf34830787cd 2023-02-09 14:55:49,542 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 14:55:49,543 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:55:49,543 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33. 2023-02-09 14:55:49,543 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 29257a2a7fe73e90adb45bdafa6efa33: Waiting for close lock at 1675954549523Running coprocessor pre-close hooks at 1675954549523Disabling compacts and flushes for region at 1675954549523Disabling writes for close at 1675954549523Writing region close event to WAL at 1675954549540 (+17 ms)Running coprocessor post-close hooks at 1675954549543 (+3 ms)Closed at 1675954549543 2023-02-09 14:55:49,544 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 29257a2a7fe73e90adb45bdafa6efa33 2023-02-09 14:55:49,544 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close c977f0f6238f5b099c4fb893ca8719f7 2023-02-09 14:55:49,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing c977f0f6238f5b099c4fb893ca8719f7, disabling compactions & flushes 2023-02-09 14:55:49,545 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,71,1675954548499.c977f0f6238f5b099c4fb893ca8719f7. 2023-02-09 14:55:49,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1675954548499.c977f0f6238f5b099c4fb893ca8719f7. 2023-02-09 14:55:49,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1675954548499.c977f0f6238f5b099c4fb893ca8719f7. after waiting 0 ms 2023-02-09 14:55:49,545 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1675954548499.c977f0f6238f5b099c4fb893ca8719f7. 2023-02-09 14:55:49,545 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=29257a2a7fe73e90adb45bdafa6efa33, regionState=CLOSED 2023-02-09 14:55:49,546 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675954549545"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954549545"}]},"ts":"1675954549545"} 2023-02-09 14:55:49,548 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-02-09 14:55:49,548 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegionEmptyDaughter/c977f0f6238f5b099c4fb893ca8719f7/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-02-09 14:55:49,548 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure 29257a2a7fe73e90adb45bdafa6efa33, server=jenkins-hbase4.apache.org,33423,1675954443133 in 171 msec 2023-02-09 14:55:49,548 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:55:49,549 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,71,1675954548499.c977f0f6238f5b099c4fb893ca8719f7. 2023-02-09 14:55:49,549 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for c977f0f6238f5b099c4fb893ca8719f7: Waiting for close lock at 1675954549544Running coprocessor pre-close hooks at 1675954549544Disabling compacts and flushes for region at 1675954549544Disabling writes for close at 1675954549545 (+1 ms)Writing region close event to WAL at 1675954549545Running coprocessor post-close hooks at 1675954549548 (+3 ms)Closed at 1675954549549 (+1 ms) 2023-02-09 14:55:49,549 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=29257a2a7fe73e90adb45bdafa6efa33, UNASSIGN in 178 msec 2023-02-09 14:55:49,550 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed c977f0f6238f5b099c4fb893ca8719f7 2023-02-09 14:55:49,550 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=c977f0f6238f5b099c4fb893ca8719f7, regionState=CLOSED 2023-02-09 14:55:49,550 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1675954548499.c977f0f6238f5b099c4fb893ca8719f7.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675954549550"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954549550"}]},"ts":"1675954549550"} 2023-02-09 14:55:49,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-02-09 14:55:49,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure c977f0f6238f5b099c4fb893ca8719f7, server=jenkins-hbase4.apache.org,33423,1675954443133 in 179 msec 2023-02-09 14:55:49,553 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=137, resume processing ppid=135 2023-02-09 14:55:49,553 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c977f0f6238f5b099c4fb893ca8719f7, UNASSIGN in 182 msec 2023-02-09 14:55:49,554 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954549554"}]},"ts":"1675954549554"} 2023-02-09 14:55:49,555 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-02-09 14:55:49,556 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-02-09 14:55:49,557 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 191 msec 2023-02-09 14:55:49,668 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:49,668 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:49,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-09 14:55:49,669 INFO [Listener at localhost/40823] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-02-09 14:55:49,669 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:49,669 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:49,670 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegionEmptyDaughter 2023-02-09 14:55:49,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 14:55:49,672 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 14:55:49,672 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:49,672 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:49,673 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 14:55:49,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-09 14:55:49,676 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:49,676 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33 2023-02-09 14:55:49,676 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/c977f0f6238f5b099c4fb893ca8719f7 2023-02-09 14:55:49,679 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/recovered.edits] 2023-02-09 14:55:49,679 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/recovered.edits] 2023-02-09 14:55:49,679 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/c977f0f6238f5b099c4fb893ca8719f7/cf1, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/c977f0f6238f5b099c4fb893ca8719f7/cf2, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/c977f0f6238f5b099c4fb893ca8719f7/recovered.edits] 2023-02-09 14:55:49,682 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/153e931814f34883bf32d3fb032365a7 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf1/153e931814f34883bf32d3fb032365a7 2023-02-09 14:55:49,682 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/17ce46f94e524b2e934c31e91a889269 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/17ce46f94e524b2e934c31e91a889269 2023-02-09 14:55:49,684 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/54d3308a08234d96a61522b5e62fdc06 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/54d3308a08234d96a61522b5e62fdc06 2023-02-09 14:55:49,685 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/9d33547d42b54f7ab70889de30e5b1e2 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/cf2/9d33547d42b54f7ab70889de30e5b1e2 2023-02-09 14:55:49,685 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/c977f0f6238f5b099c4fb893ca8719f7/recovered.edits/88.seqid to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/c977f0f6238f5b099c4fb893ca8719f7/recovered.edits/88.seqid 2023-02-09 14:55:49,685 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/6cb7d6ee6da3479c86f3a7c515a5c385 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/6cb7d6ee6da3479c86f3a7c515a5c385 2023-02-09 14:55:49,685 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/c977f0f6238f5b099c4fb893ca8719f7 2023-02-09 14:55:49,686 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/92e68d7b3a9649c8afcad149e9bf1c73 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/92e68d7b3a9649c8afcad149e9bf1c73 2023-02-09 14:55:49,687 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/recovered.edits/90.seqid to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33/recovered.edits/90.seqid 2023-02-09 14:55:49,688 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/9f2e3516efb64f778d5e4e973f55350f to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/9f2e3516efb64f778d5e4e973f55350f 2023-02-09 14:55:49,688 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/29257a2a7fe73e90adb45bdafa6efa33 2023-02-09 14:55:49,689 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/c767666c4705461b9c2d8a9fbfed0aa8 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/c767666c4705461b9c2d8a9fbfed0aa8 2023-02-09 14:55:49,690 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/f334b4fe58114b65bc82fa6b079d3340 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf1/f334b4fe58114b65bc82fa6b079d3340 2023-02-09 14:55:49,692 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/0e0587587b864889aec67ab75196bb15 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/0e0587587b864889aec67ab75196bb15 2023-02-09 14:55:49,693 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/3ed8ebfa31c946a5b358bd4fc871d7e1 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/3ed8ebfa31c946a5b358bd4fc871d7e1 2023-02-09 14:55:49,694 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/595b9bc730d7470e8da9c19eefa6848f to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/595b9bc730d7470e8da9c19eefa6848f 2023-02-09 14:55:49,695 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/6a5d53fe6f444b7c90b8959ca3f58956 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/6a5d53fe6f444b7c90b8959ca3f58956 2023-02-09 14:55:49,696 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/701d0450d86e41478a87d2c7e8635d0f to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/701d0450d86e41478a87d2c7e8635d0f 2023-02-09 14:55:49,697 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/af4200d2d0bc434ba98b6bad729e4a1b to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/af4200d2d0bc434ba98b6bad729e4a1b 2023-02-09 14:55:49,697 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/e626cbb12e0049a4bcc2cf34830787cd to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/cf2/e626cbb12e0049a4bcc2cf34830787cd 2023-02-09 14:55:49,699 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/recovered.edits/85.seqid to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd/recovered.edits/85.seqid 2023-02-09 14:55:49,700 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegionEmptyDaughter/109e2ee6b532fbae23a8a99627ce82fd 2023-02-09 14:55:49,700 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-09 14:55:49,701 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 14:55:49,703 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-02-09 14:55:49,705 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-02-09 14:55:49,705 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 14:55:49,705 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-02-09 14:55:49,705 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954549705"}]},"ts":"9223372036854775807"} 2023-02-09 14:55:49,705 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954549705"}]},"ts":"9223372036854775807"} 2023-02-09 14:55:49,706 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1675954548499.c977f0f6238f5b099c4fb893ca8719f7.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954549705"}]},"ts":"9223372036854775807"} 2023-02-09 14:55:49,707 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 14:55:49,707 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 109e2ee6b532fbae23a8a99627ce82fd, NAME => 'testSplitTableRegionEmptyDaughter,,1675954545772.109e2ee6b532fbae23a8a99627ce82fd.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 29257a2a7fe73e90adb45bdafa6efa33, NAME => 'testSplitTableRegionEmptyDaughter,,1675954548499.29257a2a7fe73e90adb45bdafa6efa33.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => c977f0f6238f5b099c4fb893ca8719f7, NAME => 'testSplitTableRegionEmptyDaughter,71,1675954548499.c977f0f6238f5b099c4fb893ca8719f7.', STARTKEY => '71', ENDKEY => ''}] 2023-02-09 14:55:49,707 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-02-09 14:55:49,707 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675954549707"}]},"ts":"9223372036854775807"} 2023-02-09 14:55:49,708 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-02-09 14:55:49,711 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-09 14:55:49,712 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 41 msec 2023-02-09 14:55:49,773 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:49,773 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:49,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-09 14:55:49,774 INFO [Listener at localhost/40823] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-02-09 14:55:49,788 INFO [Listener at localhost/40823] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=374 (was 375), OpenFileDescriptor=833 (was 833), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=85 (was 85), ProcessCount=168 (was 168), AvailableMemoryMB=2941 (was 2946) 2023-02-09 14:55:49,801 INFO [Listener at localhost/40823] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=374, OpenFileDescriptor=833, MaxFileDescriptor=60000, SystemLoadAverage=85, ProcessCount=168, AvailableMemoryMB=2941 2023-02-09 14:55:49,801 WARN [Listener at localhost/40823] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 14:55:49,801 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:49,801 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:49,803 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-09 14:55:49,803 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-02-09 14:55:49,805 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-02-09 14:55:49,806 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-02-09 14:55:49,806 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-09 14:55:49,807 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:49,808 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb empty. 2023-02-09 14:55:49,808 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:49,808 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-09 14:55:49,819 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-02-09 14:55:49,820 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7671): creating {ENCODED => 2a77ee9910df938efcf9f9f8fd3a86cb, NAME => 'testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp 2023-02-09 14:55:49,829 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:49,829 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1603): Closing 2a77ee9910df938efcf9f9f8fd3a86cb, disabling compactions & flushes 2023-02-09 14:55:49,829 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. 2023-02-09 14:55:49,829 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. 2023-02-09 14:55:49,829 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. after waiting 0 ms 2023-02-09 14:55:49,829 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. 2023-02-09 14:55:49,829 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. 2023-02-09 14:55:49,829 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1557): Region close journal for 2a77ee9910df938efcf9f9f8fd3a86cb: Waiting for close lock at 1675954549829Disabling compacts and flushes for region at 1675954549829Disabling writes for close at 1675954549829Writing region close event to WAL at 1675954549829Closed at 1675954549829 2023-02-09 14:55:49,831 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-02-09 14:55:49,832 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675954549831"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954549831"}]},"ts":"1675954549831"} 2023-02-09 14:55:49,832 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-09 14:55:49,833 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-09 14:55:49,833 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954549833"}]},"ts":"1675954549833"} 2023-02-09 14:55:49,834 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-02-09 14:55:49,836 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-02-09 14:55:49,837 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-09 14:55:49,837 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-09 14:55:49,837 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-09 14:55:49,837 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-09 14:55:49,837 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=2a77ee9910df938efcf9f9f8fd3a86cb, ASSIGN}] 2023-02-09 14:55:49,838 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=2a77ee9910df938efcf9f9f8fd3a86cb, ASSIGN 2023-02-09 14:55:49,839 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=2a77ee9910df938efcf9f9f8fd3a86cb, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,33423,1675954443133; forceNewPlan=false, retain=false 2023-02-09 14:55:49,989 INFO [jenkins-hbase4:43829] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-09 14:55:49,990 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=2a77ee9910df938efcf9f9f8fd3a86cb, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:49,990 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675954549990"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954549990"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954549990"}]},"ts":"1675954549990"} 2023-02-09 14:55:49,992 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure 2a77ee9910df938efcf9f9f8fd3a86cb, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:55:50,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. 2023-02-09 14:55:50,146 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 2a77ee9910df938efcf9f9f8fd3a86cb, NAME => 'testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb.', STARTKEY => '', ENDKEY => ''} 2023-02-09 14:55:50,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:55:50,146 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:50,146 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:50,146 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:50,146 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:50,147 INFO [StoreOpener-2a77ee9910df938efcf9f9f8fd3a86cb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:50,149 DEBUG [StoreOpener-2a77ee9910df938efcf9f9f8fd3a86cb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1 2023-02-09 14:55:50,150 DEBUG [StoreOpener-2a77ee9910df938efcf9f9f8fd3a86cb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1 2023-02-09 14:55:50,150 INFO [StoreOpener-2a77ee9910df938efcf9f9f8fd3a86cb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2a77ee9910df938efcf9f9f8fd3a86cb columnFamilyName cf1 2023-02-09 14:55:50,150 INFO [StoreOpener-2a77ee9910df938efcf9f9f8fd3a86cb-1] regionserver.HStore(310): Store=2a77ee9910df938efcf9f9f8fd3a86cb/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:50,151 INFO [StoreOpener-2a77ee9910df938efcf9f9f8fd3a86cb-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:50,152 DEBUG [StoreOpener-2a77ee9910df938efcf9f9f8fd3a86cb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2 2023-02-09 14:55:50,152 DEBUG [StoreOpener-2a77ee9910df938efcf9f9f8fd3a86cb-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2 2023-02-09 14:55:50,152 INFO [StoreOpener-2a77ee9910df938efcf9f9f8fd3a86cb-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2a77ee9910df938efcf9f9f8fd3a86cb columnFamilyName cf2 2023-02-09 14:55:50,153 INFO [StoreOpener-2a77ee9910df938efcf9f9f8fd3a86cb-1] regionserver.HStore(310): Store=2a77ee9910df938efcf9f9f8fd3a86cb/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:50,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:50,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:50,156 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:55:50,157 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:50,159 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-09 14:55:50,159 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 2a77ee9910df938efcf9f9f8fd3a86cb; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71982233, jitterRate=0.07261885702610016}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:55:50,159 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 2a77ee9910df938efcf9f9f8fd3a86cb: Running coprocessor pre-open hook at 1675954550146Writing region info on filesystem at 1675954550146Initializing all the Stores at 1675954550147 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954550147Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954550147Cleaning up temporary data from old regions at 1675954550155 (+8 ms)Cleaning up detritus from prior splits at 1675954550155Running coprocessor post-open hooks at 1675954550159 (+4 ms)Region opened successfully at 1675954550159 2023-02-09 14:55:50,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb., pid=143, masterSystemTime=1675954550143 2023-02-09 14:55:50,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. 2023-02-09 14:55:50,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. 2023-02-09 14:55:50,162 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=2a77ee9910df938efcf9f9f8fd3a86cb, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,162 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675954550162"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954550162"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954550162"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954550162"}]},"ts":"1675954550162"} 2023-02-09 14:55:50,164 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-02-09 14:55:50,164 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure 2a77ee9910df938efcf9f9f8fd3a86cb, server=jenkins-hbase4.apache.org,33423,1675954443133 in 172 msec 2023-02-09 14:55:50,166 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-02-09 14:55:50,167 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=2a77ee9910df938efcf9f9f8fd3a86cb, ASSIGN in 327 msec 2023-02-09 14:55:50,167 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-09 14:55:50,167 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954550167"}]},"ts":"1675954550167"} 2023-02-09 14:55:50,168 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-02-09 14:55:50,171 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-02-09 14:55:50,175 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 367 msec 2023-02-09 14:55:50,310 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:50,311 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:50,311 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:50,312 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-09 14:55:50,312 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-09 14:55:50,312 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:50,312 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-09 14:55:50,312 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-09 14:55:50,313 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,313 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,314 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:50,314 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:50,314 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:50,314 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:50,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:50,314 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,314 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:50,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:50,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:50,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:50,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,315 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,315 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:50,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,315 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,315 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,315 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:50,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:50,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:50,316 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 14:55:50,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,316 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 14:55:50,316 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 14:55:50,316 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 14:55:50,316 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 14:55:50,316 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:50,316 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 14:55:50,316 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 14:55:50,316 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 14:55:50,316 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 14:55:50,316 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 14:55:50,316 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 14:55:50,316 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 14:55:50,316 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:50,316 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:50,316 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 14:55:50,316 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 14:55:50,316 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 14:55:50,317 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 14:55:50,317 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:50,317 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:50,317 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. 2023-02-09 14:55:50,317 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. started... 2023-02-09 14:55:50,318 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2a77ee9910df938efcf9f9f8fd3a86cb 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-09 14:55:50,318 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,319 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,319 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,319 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:50,320 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,320 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:50,326 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf1/bb3bd5cf3bfb412f8a7efd2ffeefec85 2023-02-09 14:55:50,337 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf2/717cc38c7361464593092579c4a3d6ba 2023-02-09 14:55:50,340 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf1/bb3bd5cf3bfb412f8a7efd2ffeefec85 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/bb3bd5cf3bfb412f8a7efd2ffeefec85 2023-02-09 14:55:50,343 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/bb3bd5cf3bfb412f8a7efd2ffeefec85, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 14:55:50,344 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf2/717cc38c7361464593092579c4a3d6ba as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/717cc38c7361464593092579c4a3d6ba 2023-02-09 14:55:50,347 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/717cc38c7361464593092579c4a3d6ba, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 14:55:50,347 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 2a77ee9910df938efcf9f9f8fd3a86cb in 29ms, sequenceid=6, compaction requested=false 2023-02-09 14:55:50,347 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-02-09 14:55:50,348 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2a77ee9910df938efcf9f9f8fd3a86cb: 2023-02-09 14:55:50,348 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. 2023-02-09 14:55:50,348 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:50,348 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:50,348 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,348 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 14:55:50,348 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 14:55:50,349 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,349 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:50,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:50,350 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,350 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:50,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:50,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:50,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:50,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:50,352 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-09 14:55:50,352 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@cdc877c[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:50,352 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-09 14:55:50,352 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,353 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,353 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,353 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,354 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 14:55:50,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,354 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,354 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:50,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,354 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 14:55:50,354 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 14:55:50,354 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 14:55:50,354 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 14:55:50,354 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:55:50,354 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 14:55:50,354 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:55:50,354 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:55:50,359 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 14:55:50,359 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,359 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 14:55:50,359 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 14:55:50,359 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:50,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:50,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:50,359 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:50,359 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 14:55:50,359 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:50,359 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 14:55:50,359 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 14:55:50,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:50,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:50,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:50,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:50,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:50,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33423,1675954443133': 2023-02-09 14:55:50,362 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-09 14:55:50,362 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-09 14:55:50,363 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:50,363 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:50,363 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-09 14:55:50,363 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:50,364 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,364 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,364 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,364 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:50,364 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,364 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:50,364 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:50,364 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,365 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:50,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:50,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:50,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:50,365 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:50,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:50,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:50,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:50,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:50,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,365 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:50,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:50,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:50,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,367 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,367 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:50,367 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:50,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,370 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,370 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,370 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:50,370 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:50,370 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:50,370 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-09 14:55:50,370 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:50,370 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,370 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:50,370 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:50,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:50,370 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:50,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:50,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:50,370 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-09 14:55:50,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:50,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:50,370 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:50,371 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:50,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:50,370 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:50,371 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,371 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:50,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:50,371 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:50,371 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,371 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:50,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:50,371 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,371 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,471 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:50,471 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:50,471 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:50,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:50,480 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:50,480 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:50,481 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:50,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:50,482 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-09 14:55:50,482 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-09 14:55:50,482 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:50,482 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-09 14:55:50,482 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-09 14:55:50,483 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,483 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,484 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,484 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:50,484 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:50,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:50,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:50,484 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:50,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:50,484 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,484 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:50,484 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:50,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:50,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:50,485 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:50,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,485 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,485 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 14:55:50,485 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,485 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 14:55:50,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:50,485 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 14:55:50,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:50,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,486 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 14:55:50,486 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 14:55:50,485 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 14:55:50,486 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:50,486 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 14:55:50,486 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 14:55:50,486 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 14:55:50,486 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 14:55:50,486 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 14:55:50,486 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 14:55:50,486 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 14:55:50,486 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:50,486 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 14:55:50,486 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 14:55:50,486 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 14:55:50,486 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:50,486 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 14:55:50,486 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 14:55:50,486 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 14:55:50,486 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:50,486 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. 2023-02-09 14:55:50,486 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. started... 2023-02-09 14:55:50,486 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:50,486 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2a77ee9910df938efcf9f9f8fd3a86cb 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:55:50,488 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,488 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,488 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:50,489 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,489 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,489 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:50,498 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf1/e227650000754bd2aabae7f134f5954a 2023-02-09 14:55:50,511 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf2/6a12cec5b8ee4a48a36e6ab7d902f86a 2023-02-09 14:55:50,515 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf1/e227650000754bd2aabae7f134f5954a as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/e227650000754bd2aabae7f134f5954a 2023-02-09 14:55:50,518 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/e227650000754bd2aabae7f134f5954a, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 14:55:50,519 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf2/6a12cec5b8ee4a48a36e6ab7d902f86a as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/6a12cec5b8ee4a48a36e6ab7d902f86a 2023-02-09 14:55:50,522 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/6a12cec5b8ee4a48a36e6ab7d902f86a, entries=10, sequenceid=19, filesize=5.0 K 2023-02-09 14:55:50,522 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2a77ee9910df938efcf9f9f8fd3a86cb in 36ms, sequenceid=19, compaction requested=false 2023-02-09 14:55:50,522 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2a77ee9910df938efcf9f9f8fd3a86cb: 2023-02-09 14:55:50,523 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. 2023-02-09 14:55:50,523 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:50,523 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:50,523 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,523 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 14:55:50,523 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 14:55:50,524 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,524 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:50,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:50,524 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,524 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:50,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:50,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:50,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:50,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:50,526 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-09 14:55:50,526 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6d428165[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:50,526 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-09 14:55:50,526 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,527 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,527 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,527 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 14:55:50,527 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,527 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:50,527 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 14:55:50,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,527 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:55:50,528 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 14:55:50,528 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 14:55:50,527 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,528 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:55:50,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,528 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 14:55:50,528 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 14:55:50,528 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:55:50,530 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 14:55:50,530 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:50,530 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,530 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 14:55:50,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:50,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:50,531 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 14:55:50,531 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:50,531 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 14:55:50,531 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 14:55:50,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:50,531 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:50,531 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 14:55:50,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:50,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:50,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:50,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:50,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33423,1675954443133': 2023-02-09 14:55:50,534 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-09 14:55:50,534 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-09 14:55:50,534 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:50,534 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:50,534 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-09 14:55:50,534 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:50,536 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,536 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,536 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,536 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,536 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:50,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,536 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:50,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,536 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:50,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:50,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,536 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:50,536 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:50,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:50,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:50,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:50,536 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:50,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:50,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:50,537 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:50,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:50,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:50,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,538 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,538 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:50,539 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:50,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,545 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,545 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:50,545 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,545 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:50,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:50,545 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:50,545 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:50,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:50,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:50,545 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,545 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:50,545 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,545 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:50,545 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,545 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-09 14:55:50,545 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:50,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:50,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:50,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:50,545 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:50,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:50,546 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-09 14:55:50,545 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,546 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:50,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:50,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:50,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:50,546 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,546 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:50,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:50,546 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,546 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,546 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,646 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:50,646 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:50,646 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:50,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:50,656 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:50,656 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:50,657 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:50,658 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:50,658 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-09 14:55:50,658 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-09 14:55:50,658 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:50,658 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-09 14:55:50,658 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-09 14:55:50,659 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,659 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,660 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:50,660 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:50,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:50,660 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:50,660 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:50,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:50,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:50,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:50,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:50,660 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,660 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:50,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,661 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,661 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,661 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:50,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:50,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:50,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,661 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 14:55:50,661 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 14:55:50,661 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 14:55:50,661 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 14:55:50,662 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 14:55:50,662 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 14:55:50,662 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 14:55:50,662 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 14:55:50,662 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 14:55:50,662 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 14:55:50,662 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 14:55:50,662 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:50,662 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 14:55:50,662 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:50,662 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:50,662 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 14:55:50,662 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 14:55:50,662 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 14:55:50,662 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 14:55:50,662 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 14:55:50,662 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 14:55:50,662 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 14:55:50,662 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:50,662 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:50,662 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. 2023-02-09 14:55:50,662 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. started... 2023-02-09 14:55:50,663 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2a77ee9910df938efcf9f9f8fd3a86cb 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:55:50,667 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,667 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,667 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,667 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,667 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:50,667 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:50,672 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf1/1535dc47022d4e169cefe91a535e1ea5 2023-02-09 14:55:50,683 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf2/f74cb0ae43544948972923924685b253 2023-02-09 14:55:50,686 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf1/1535dc47022d4e169cefe91a535e1ea5 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/1535dc47022d4e169cefe91a535e1ea5 2023-02-09 14:55:50,689 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/1535dc47022d4e169cefe91a535e1ea5, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 14:55:50,689 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf2/f74cb0ae43544948972923924685b253 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/f74cb0ae43544948972923924685b253 2023-02-09 14:55:50,692 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/f74cb0ae43544948972923924685b253, entries=10, sequenceid=32, filesize=5.0 K 2023-02-09 14:55:50,692 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2a77ee9910df938efcf9f9f8fd3a86cb in 30ms, sequenceid=32, compaction requested=true 2023-02-09 14:55:50,693 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2a77ee9910df938efcf9f9f8fd3a86cb: 2023-02-09 14:55:50,693 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. 2023-02-09 14:55:50,693 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:50,693 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:50,693 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,693 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 14:55:50,693 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 14:55:50,694 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,694 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:50,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:50,694 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,694 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:50,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:50,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:50,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:50,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:50,696 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-09 14:55:50,696 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@38fdfa10[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:50,696 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-09 14:55:50,696 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,697 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,697 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,697 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,697 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,697 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:50,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,697 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 14:55:50,697 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 14:55:50,697 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 14:55:50,697 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 14:55:50,697 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:55:50,697 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:55:50,697 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 14:55:50,697 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 14:55:50,697 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:55:50,698 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 14:55:50,698 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 14:55:50,698 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,698 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:50,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:50,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:50,698 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:50,699 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 14:55:50,699 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 14:55:50,700 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 14:55:50,700 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:50,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:50,700 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 14:55:50,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:50,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:50,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:50,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:50,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33423,1675954443133': 2023-02-09 14:55:50,702 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-09 14:55:50,702 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-09 14:55:50,703 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:50,703 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:50,703 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-09 14:55:50,703 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:50,704 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,704 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:50,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:50,704 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,704 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,704 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:50,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:50,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,704 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:50,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:50,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:50,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,704 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:50,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:50,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:50,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:50,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:50,704 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:50,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:50,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,705 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:50,705 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:50,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:50,707 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,707 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,707 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,710 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:50,710 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:50,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:50,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:50,710 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-09 14:55:50,710 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:50,710 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:50,710 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:50,710 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:50,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:50,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:50,710 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:50,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:50,710 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-09 14:55:50,710 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:50,710 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:50,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:50,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:50,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:50,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:50,710 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,710 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:50,711 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,711 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:50,711 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:50,711 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,711 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,711 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,711 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,711 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,711 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,811 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:50,811 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:50,811 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:50,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:50,821 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:50,821 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:50,822 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:50,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:50,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-09 14:55:50,823 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-09 14:55:50,823 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:50,823 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-09 14:55:50,823 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-09 14:55:50,824 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,824 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,825 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,825 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:50,825 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:50,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:50,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:50,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:50,825 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:50,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:50,826 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,826 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:50,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:50,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:50,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,826 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,826 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,826 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:50,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:50,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:50,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,827 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 14:55:50,827 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 14:55:50,827 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 14:55:50,827 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 14:55:50,827 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 14:55:50,827 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 14:55:50,827 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 14:55:50,827 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:50,827 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 14:55:50,827 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 14:55:50,827 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 14:55:50,827 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 14:55:50,828 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:50,827 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 14:55:50,828 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 14:55:50,828 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 14:55:50,828 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 14:55:50,827 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 14:55:50,828 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 14:55:50,828 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 14:55:50,828 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:50,828 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 14:55:50,828 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:50,828 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:50,828 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. 2023-02-09 14:55:50,828 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. started... 2023-02-09 14:55:50,828 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2a77ee9910df938efcf9f9f8fd3a86cb 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:55:50,830 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,830 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,830 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,830 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:50,830 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,830 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:50,839 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf1/e4c94818f8d2490bb3bac04b76c3da0d 2023-02-09 14:55:50,849 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf2/a4c9589bbd744933aa32467915dced43 2023-02-09 14:55:50,852 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf1/e4c94818f8d2490bb3bac04b76c3da0d as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/e4c94818f8d2490bb3bac04b76c3da0d 2023-02-09 14:55:50,854 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/e4c94818f8d2490bb3bac04b76c3da0d, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 14:55:50,855 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf2/a4c9589bbd744933aa32467915dced43 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/a4c9589bbd744933aa32467915dced43 2023-02-09 14:55:50,858 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/a4c9589bbd744933aa32467915dced43, entries=10, sequenceid=45, filesize=5.0 K 2023-02-09 14:55:50,859 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2a77ee9910df938efcf9f9f8fd3a86cb in 31ms, sequenceid=45, compaction requested=true 2023-02-09 14:55:50,859 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2a77ee9910df938efcf9f9f8fd3a86cb: 2023-02-09 14:55:50,859 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. 2023-02-09 14:55:50,859 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:50,859 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:50,860 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,860 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 14:55:50,860 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 14:55:50,861 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,862 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,862 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:50,866 ERROR [BP-2069929948-172.31.14.131-1675954438937 heartbeating to localhost/127.0.0.1:38049] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2069929948-172.31.14.131-1675954438937 (Datanode Uuid 30e687f1-c071-43d7-89db-78c265b7458a) service to localhost/127.0.0.1:38049 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-09 14:55:50,868 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:50,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:50,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:50,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:50,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:50,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:50,872 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-09 14:55:50,872 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@73daaa16[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:50,872 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-09 14:55:50,872 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,873 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,873 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,873 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,874 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 14:55:50,874 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 14:55:50,874 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:55:50,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,874 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,874 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 14:55:50,874 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 14:55:50,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,874 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:55:50,874 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 14:55:50,874 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 14:55:50,874 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:55:50,874 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:50,875 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 14:55:50,875 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:50,875 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 14:55:50,876 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,876 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 14:55:50,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:50,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:50,876 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:50,876 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 14:55:50,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:50,878 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 14:55:50,878 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:50,878 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 14:55:50,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:50,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:50,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:50,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:50,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33423,1675954443133': 2023-02-09 14:55:50,885 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-09 14:55:50,885 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-09 14:55:50,886 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:50,886 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:50,886 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-09 14:55:50,886 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:50,887 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,887 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,887 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,887 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:50,887 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,887 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:50,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,887 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:50,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:50,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:50,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:50,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:50,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:50,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:50,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:50,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:50,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:50,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,888 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:50,888 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:50,889 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:50,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:50,890 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:50,891 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,891 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,899 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:50,899 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:50,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:50,899 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:50,899 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:50,899 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:50,899 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:50,899 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-09 14:55:50,899 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:50,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:50,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:50,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:50,899 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:50,900 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-09 14:55:50,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:50,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:50,899 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:50,900 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:50,900 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:50,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:50,900 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:50,900 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,900 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,900 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:50,901 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:50,901 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,901 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:50,901 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:50,901 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:50,901 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:50,901 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:50,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:51,000 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:51,000 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:51,001 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:51,001 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:51,010 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:51,010 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:51,011 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:51,012 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:51,012 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-09 14:55:51,012 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-09 14:55:51,012 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:51,012 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-09 14:55:51,012 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-09 14:55:51,013 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,013 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:51,014 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:51,014 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:51,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:51,014 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,014 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:51,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:51,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:51,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:51,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:51,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:51,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:51,014 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:51,014 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:51,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:51,014 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,015 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,015 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:51,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:51,015 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 14:55:51,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:51,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:51,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:51,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:51,015 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 14:55:51,015 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 14:55:51,015 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 14:55:51,015 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 14:55:51,015 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 14:55:51,015 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 14:55:51,015 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 14:55:51,015 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 14:55:51,015 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 14:55:51,015 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:51,015 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:51,015 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 14:55:51,016 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 14:55:51,016 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 14:55:51,016 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:51,016 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 14:55:51,016 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 14:55:51,016 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:51,016 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:51,016 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. 2023-02-09 14:55:51,016 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 14:55:51,016 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. started... 2023-02-09 14:55:51,016 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 14:55:51,016 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 14:55:51,016 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 14:55:51,016 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2a77ee9910df938efcf9f9f8fd3a86cb 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:55:51,017 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,017 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,018 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,018 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,018 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:51,018 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:51,025 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf1/16b9406f9d754f5b8650053c382380fb 2023-02-09 14:55:51,034 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf2/bc62a2c7a24c49be874e2f455259580e 2023-02-09 14:55:51,037 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf1/16b9406f9d754f5b8650053c382380fb as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/16b9406f9d754f5b8650053c382380fb 2023-02-09 14:55:51,039 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/16b9406f9d754f5b8650053c382380fb, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 14:55:51,040 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf2/bc62a2c7a24c49be874e2f455259580e as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/bc62a2c7a24c49be874e2f455259580e 2023-02-09 14:55:51,043 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/bc62a2c7a24c49be874e2f455259580e, entries=10, sequenceid=58, filesize=5.0 K 2023-02-09 14:55:51,043 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2a77ee9910df938efcf9f9f8fd3a86cb in 27ms, sequenceid=58, compaction requested=true 2023-02-09 14:55:51,043 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2a77ee9910df938efcf9f9f8fd3a86cb: 2023-02-09 14:55:51,043 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. 2023-02-09 14:55:51,043 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:51,043 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:51,043 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,043 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 14:55:51,043 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 14:55:51,045 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,045 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:51,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:51,045 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,045 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:51,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:51,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:51,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:51,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:51,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:51,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:51,046 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-09 14:55:51,047 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b553476[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:51,047 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-09 14:55:51,047 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,053 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,053 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,053 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,053 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 14:55:51,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,053 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 14:55:51,053 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,053 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:51,053 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 14:55:51,053 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 14:55:51,053 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:55:51,053 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:55:51,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,053 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 14:55:51,053 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 14:55:51,053 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:55:51,054 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 14:55:51,054 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,054 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 14:55:51,054 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:51,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,054 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:51,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:51,055 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 14:55:51,055 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 14:55:51,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:51,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:51,055 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 14:55:51,055 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:51,055 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 14:55:51,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:51,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:51,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:51,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:51,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:51,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:51,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:51,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:51,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33423,1675954443133': 2023-02-09 14:55:51,058 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-09 14:55:51,058 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-09 14:55:51,058 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:51,058 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:51,058 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-09 14:55:51,058 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:51,059 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,059 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,059 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,059 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,059 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:51,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,059 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:51,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:51,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:51,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,060 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:51,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:51,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:51,059 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:51,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:51,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:51,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:51,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,060 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:51,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:51,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:51,060 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:51,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:51,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:51,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:51,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:51,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:51,062 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:51,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:51,062 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:51,062 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:51,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:51,065 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:51,065 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:51,065 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,065 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:51,065 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:51,065 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:51,065 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:51,065 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:51,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:51,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:51,066 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:51,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:51,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:51,065 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:51,065 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-09 14:55:51,066 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:51,065 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:51,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:51,066 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:51,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:51,066 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:51,066 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,066 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,066 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:51,066 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-09 14:55:51,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:51,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:51,066 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:51,066 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:51,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:51,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:51,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:51,067 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,167 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:51,167 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:51,167 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:51,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:51,178 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:51,178 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:51,178 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-09 14:55:51,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-09 14:55:51,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-09 14:55:51,180 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-09 14:55:51,180 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:51,180 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-09 14:55:51,180 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-09 14:55:51,180 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,180 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:51,181 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:51,181 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:51,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:51,181 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:51,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:51,181 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,181 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:51,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:51,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:51,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:51,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:51,182 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,182 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-09 14:55:51,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:51,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:51,182 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,182 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,182 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:51,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:51,183 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 14:55:51,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:51,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:51,183 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 14:55:51,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-09 14:55:51,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:51,183 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 14:55:51,183 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-09 14:55:51,183 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 14:55:51,183 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 14:55:51,183 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 14:55:51,183 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 14:55:51,183 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 14:55:51,183 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:51,183 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-09 14:55:51,183 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:51,183 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-09 14:55:51,183 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 14:55:51,183 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-09 14:55:51,183 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-09 14:55:51,183 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 14:55:51,183 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 14:55:51,183 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-09 14:55:51,184 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33553,1675954443180' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 14:55:51,183 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 14:55:51,184 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,41077,1675954443212' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 14:55:51,184 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-09 14:55:51,184 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-09 14:55:51,184 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. 2023-02-09 14:55:51,184 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. started... 2023-02-09 14:55:51,184 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 2a77ee9910df938efcf9f9f8fd3a86cb 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-09 14:55:51,185 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,185 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,185 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,186 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:51,186 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,186 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:51,195 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf1/3ddaee433454488594c6ffdf78491e29 2023-02-09 14:55:51,205 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf2/5bb552d633564dbbac7522449c5dc66f 2023-02-09 14:55:51,208 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf1/3ddaee433454488594c6ffdf78491e29 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/3ddaee433454488594c6ffdf78491e29 2023-02-09 14:55:51,210 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/3ddaee433454488594c6ffdf78491e29, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 14:55:51,211 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf2/5bb552d633564dbbac7522449c5dc66f as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/5bb552d633564dbbac7522449c5dc66f 2023-02-09 14:55:51,214 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/5bb552d633564dbbac7522449c5dc66f, entries=10, sequenceid=71, filesize=5.0 K 2023-02-09 14:55:51,215 INFO [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 2a77ee9910df938efcf9f9f8fd3a86cb in 31ms, sequenceid=71, compaction requested=true 2023-02-09 14:55:51,215 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 2a77ee9910df938efcf9f9f8fd3a86cb: 2023-02-09 14:55:51,215 DEBUG [rs(jenkins-hbase4.apache.org,33423,1675954443133)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. 2023-02-09 14:55:51,215 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-09 14:55:51,215 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-09 14:55:51,215 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,215 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-09 14:55:51,215 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-09 14:55:51,217 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,217 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:51,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:51,217 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,217 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-09 14:55:51,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:51,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:51,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:51,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:51,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:51,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:51,219 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,33423,1675954443133' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-09 14:55:51,219 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@62e5c7d0[Count = 0] remaining members to acquire global barrier 2023-02-09 14:55:51,219 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-09 14:55:51,219 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,221 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,221 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,221 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,221 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,221 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-09 14:55:51,221 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 14:55:51,221 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 14:55:51,222 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33423,1675954443133' in zk 2023-02-09 14:55:51,221 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 14:55:51,222 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 14:55:51,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,222 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,33553,1675954443180' in zk 2023-02-09 14:55:51,222 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-09 14:55:51,222 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-09 14:55:51,222 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,41077,1675954443212' in zk 2023-02-09 14:55:51,225 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,225 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 14:55:51,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:51,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:51,225 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:51,225 DEBUG [member: 'jenkins-hbase4.apache.org,33423,1675954443133' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 14:55:51,226 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 14:55:51,226 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:51,226 DEBUG [member: 'jenkins-hbase4.apache.org,33553,1675954443180' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 14:55:51,228 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-09 14:55:51,228 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:51,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:51,228 DEBUG [member: 'jenkins-hbase4.apache.org,41077,1675954443212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-09 14:55:51,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:51,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:51,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:51,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:51,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:51,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:51,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:51,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:51,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,33423,1675954443133': 2023-02-09 14:55:51,230 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,33423,1675954443133' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-09 14:55:51,230 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-09 14:55:51,231 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-09 14:55:51,231 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-09 14:55:51,231 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-09 14:55:51,231 INFO [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-09 14:55:51,232 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,232 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,232 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-09 14:55:51,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-09 14:55:51,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,232 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:51,232 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,232 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:51,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,232 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:51,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:51,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-09 14:55:51,233 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:51,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:51,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:51,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:51,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:51,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:51,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,233 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:51,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,234 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:51,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-09 14:55:51,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:51,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:51,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:51,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-09 14:55:51,235 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:51,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-09 14:55:51,235 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:51,236 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:51,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:51,238 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:51,238 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:51,238 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:51,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:51,238 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:51,238 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-09 14:55:51,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:51,238 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-09 14:55:51,238 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-09 14:55:51,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:51,238 DEBUG [(jenkins-hbase4.apache.org,43829,1675954441993)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-09 14:55:51,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-09 14:55:51,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:51,238 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:51,238 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:51,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-09 14:55:51,238 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,239 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-09 14:55:51,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:51,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:51,238 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-09 14:55:51,239 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-09 14:55:51,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:51,239 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:51,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-09 14:55:51,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:51,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-09 14:55:51,239 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-09 14:55:51,239 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:51,239 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,239 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,239 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:51,239 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-09 14:55:51,239 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-09 14:55:51,339 DEBUG [Listener at localhost/40823] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-09 14:55:51,339 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:51,339 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:51,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-09 14:55:51,347 DEBUG [Listener at localhost/40823] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,347 DEBUG [Listener at localhost/40823] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=2a77ee9910df938efcf9f9f8fd3a86cb, daughterA=d059c57c2a212eb451f423b021c9a327, daughterB=9622b3496718ce3da4db965247f3b82d 2023-02-09 14:55:51,348 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=2a77ee9910df938efcf9f9f8fd3a86cb, daughterA=d059c57c2a212eb451f423b021c9a327, daughterB=9622b3496718ce3da4db965247f3b82d 2023-02-09 14:55:51,349 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=2a77ee9910df938efcf9f9f8fd3a86cb, daughterA=d059c57c2a212eb451f423b021c9a327, daughterB=9622b3496718ce3da4db965247f3b82d 2023-02-09 14:55:51,349 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=2a77ee9910df938efcf9f9f8fd3a86cb, daughterA=d059c57c2a212eb451f423b021c9a327, daughterB=9622b3496718ce3da4db965247f3b82d 2023-02-09 14:55:51,350 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=2a77ee9910df938efcf9f9f8fd3a86cb, UNASSIGN}] 2023-02-09 14:55:51,351 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=2a77ee9910df938efcf9f9f8fd3a86cb, UNASSIGN 2023-02-09 14:55:51,351 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=2a77ee9910df938efcf9f9f8fd3a86cb, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:51,351 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675954551351"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954551351"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954551351"}]},"ts":"1675954551351"} 2023-02-09 14:55:51,352 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure 2a77ee9910df938efcf9f9f8fd3a86cb, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:55:51,504 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,504 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 2a77ee9910df938efcf9f9f8fd3a86cb, disabling compactions & flushes 2023-02-09 14:55:51,504 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. 2023-02-09 14:55:51,504 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. 2023-02-09 14:55:51,504 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. after waiting 0 ms 2023-02-09 14:55:51,504 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. 2023-02-09 14:55:51,504 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 2a77ee9910df938efcf9f9f8fd3a86cb 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-09 14:55:51,514 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf1/0bde08697e1442e593d3af4eb18f2f25 2023-02-09 14:55:51,526 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf2/193b42278c19450c9951535900a995cf 2023-02-09 14:55:51,529 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf1/0bde08697e1442e593d3af4eb18f2f25 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/0bde08697e1442e593d3af4eb18f2f25 2023-02-09 14:55:51,533 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/0bde08697e1442e593d3af4eb18f2f25, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 14:55:51,533 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/.tmp/cf2/193b42278c19450c9951535900a995cf as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/193b42278c19450c9951535900a995cf 2023-02-09 14:55:51,537 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/193b42278c19450c9951535900a995cf, entries=8, sequenceid=82, filesize=5.0 K 2023-02-09 14:55:51,537 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 2a77ee9910df938efcf9f9f8fd3a86cb in 33ms, sequenceid=82, compaction requested=true 2023-02-09 14:55:51,541 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-09 14:55:51,542 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:55:51,542 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. 2023-02-09 14:55:51,542 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 2a77ee9910df938efcf9f9f8fd3a86cb: Waiting for close lock at 1675954551504Running coprocessor pre-close hooks at 1675954551504Disabling compacts and flushes for region at 1675954551504Disabling writes for close at 1675954551504Obtaining lock to block concurrent updates at 1675954551504Preparing flush snapshotting stores in 2a77ee9910df938efcf9f9f8fd3a86cb at 1675954551504Finished memstore snapshotting testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1675954551505 (+1 ms)Flushing stores of testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. at 1675954551505Flushing 2a77ee9910df938efcf9f9f8fd3a86cb/cf1: creating writer at 1675954551506 (+1 ms)Flushing 2a77ee9910df938efcf9f9f8fd3a86cb/cf1: appending metadata at 1675954551508 (+2 ms)Flushing 2a77ee9910df938efcf9f9f8fd3a86cb/cf1: closing flushed file at 1675954551508Flushing 2a77ee9910df938efcf9f9f8fd3a86cb/cf2: creating writer at 1675954551517 (+9 ms)Flushing 2a77ee9910df938efcf9f9f8fd3a86cb/cf2: appending metadata at 1675954551519 (+2 ms)Flushing 2a77ee9910df938efcf9f9f8fd3a86cb/cf2: closing flushed file at 1675954551519Flushing 2a77ee9910df938efcf9f9f8fd3a86cb/cf1: reopening flushed file at 1675954551530 (+11 ms)Flushing 2a77ee9910df938efcf9f9f8fd3a86cb/cf2: reopening flushed file at 1675954551534 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 2a77ee9910df938efcf9f9f8fd3a86cb in 33ms, sequenceid=82, compaction requested=true at 1675954551537 (+3 ms)Writing region close event to WAL at 1675954551539 (+2 ms)Running coprocessor post-close hooks at 1675954551542 (+3 ms)Closed at 1675954551542 2023-02-09 14:55:51,543 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,544 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=2a77ee9910df938efcf9f9f8fd3a86cb, regionState=CLOSED 2023-02-09 14:55:51,544 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675954551544"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954551544"}]},"ts":"1675954551544"} 2023-02-09 14:55:51,546 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-02-09 14:55:51,546 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure 2a77ee9910df938efcf9f9f8fd3a86cb, server=jenkins-hbase4.apache.org,33423,1675954443133 in 193 msec 2023-02-09 14:55:51,547 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-02-09 14:55:51,547 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=2a77ee9910df938efcf9f9f8fd3a86cb, UNASSIGN in 196 msec 2023-02-09 14:55:51,556 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=2a77ee9910df938efcf9f9f8fd3a86cb, threads=14 2023-02-09 14:55:51,556 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/193b42278c19450c9951535900a995cf for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,556 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/5bb552d633564dbbac7522449c5dc66f for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,556 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/6a12cec5b8ee4a48a36e6ab7d902f86a for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,556 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/717cc38c7361464593092579c4a3d6ba for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,556 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/a4c9589bbd744933aa32467915dced43 for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,556 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/bc62a2c7a24c49be874e2f455259580e for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,556 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/f74cb0ae43544948972923924685b253 for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,557 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/0bde08697e1442e593d3af4eb18f2f25 for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,557 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/1535dc47022d4e169cefe91a535e1ea5 for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,557 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/16b9406f9d754f5b8650053c382380fb for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,557 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/3ddaee433454488594c6ffdf78491e29 for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,557 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/bb3bd5cf3bfb412f8a7efd2ffeefec85 for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,557 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/e227650000754bd2aabae7f134f5954a for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,557 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/e4c94818f8d2490bb3bac04b76c3da0d for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,669 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/717cc38c7361464593092579c4a3d6ba for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,676 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/5bb552d633564dbbac7522449c5dc66f for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,686 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/e227650000754bd2aabae7f134f5954a for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,687 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/0bde08697e1442e593d3af4eb18f2f25 for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,698 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/f74cb0ae43544948972923924685b253 for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,708 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/6a12cec5b8ee4a48a36e6ab7d902f86a for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,722 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/16b9406f9d754f5b8650053c382380fb for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,722 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/bb3bd5cf3bfb412f8a7efd2ffeefec85 for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,722 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/193b42278c19450c9951535900a995cf for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,728 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/1535dc47022d4e169cefe91a535e1ea5 for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,729 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/e4c94818f8d2490bb3bac04b76c3da0d for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,729 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/bc62a2c7a24c49be874e2f455259580e for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:51,731 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/a4c9589bbd744933aa32467915dced43 for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,066 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/3ddaee433454488594c6ffdf78491e29 for region: 2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,066 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region 2a77ee9910df938efcf9f9f8fd3a86cb Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-09 14:55:52,095 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 14:55:52,097 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-09 14:55:52,098 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1675954552098"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1675954552098"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1675954552098"}]},"ts":"1675954552098"} 2023-02-09 14:55:52,098 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675954552098"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954552098"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954552098"}]},"ts":"1675954552098"} 2023-02-09 14:55:52,098 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675954552098"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954552098"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954552098"}]},"ts":"1675954552098"} 2023-02-09 14:55:52,100 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=d059c57c2a212eb451f423b021c9a327, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=9622b3496718ce3da4db965247f3b82d, ASSIGN}] 2023-02-09 14:55:52,101 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=9622b3496718ce3da4db965247f3b82d, ASSIGN 2023-02-09 14:55:52,101 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=9622b3496718ce3da4db965247f3b82d, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33423,1675954443133; forceNewPlan=false, retain=false 2023-02-09 14:55:52,102 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=d059c57c2a212eb451f423b021c9a327, ASSIGN 2023-02-09 14:55:52,102 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=d059c57c2a212eb451f423b021c9a327, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,33423,1675954443133; forceNewPlan=false, retain=false 2023-02-09 14:55:52,252 INFO [jenkins-hbase4:43829] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-09 14:55:52,253 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=d059c57c2a212eb451f423b021c9a327, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:52,253 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675954552253"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954552253"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954552253"}]},"ts":"1675954552253"} 2023-02-09 14:55:52,255 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=147, state=RUNNABLE; OpenRegionProcedure d059c57c2a212eb451f423b021c9a327, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:55:52,256 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=9622b3496718ce3da4db965247f3b82d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:52,257 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675954552256"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954552256"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954552256"}]},"ts":"1675954552256"} 2023-02-09 14:55:52,257 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=148, state=RUNNABLE; OpenRegionProcedure 9622b3496718ce3da4db965247f3b82d, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:55:52,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327. 2023-02-09 14:55:52,408 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d059c57c2a212eb451f423b021c9a327, NAME => 'testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.', STARTKEY => '', ENDKEY => '41'} 2023-02-09 14:55:52,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:55:52,409 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion d059c57c2a212eb451f423b021c9a327 2023-02-09 14:55:52,409 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:52,409 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for d059c57c2a212eb451f423b021c9a327 2023-02-09 14:55:52,409 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for d059c57c2a212eb451f423b021c9a327 2023-02-09 14:55:52,410 INFO [StoreOpener-d059c57c2a212eb451f423b021c9a327-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d059c57c2a212eb451f423b021c9a327 2023-02-09 14:55:52,410 DEBUG [StoreOpener-d059c57c2a212eb451f423b021c9a327-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1 2023-02-09 14:55:52,411 DEBUG [StoreOpener-d059c57c2a212eb451f423b021c9a327-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1 2023-02-09 14:55:52,411 INFO [StoreOpener-d059c57c2a212eb451f423b021c9a327-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d059c57c2a212eb451f423b021c9a327 columnFamilyName cf1 2023-02-09 14:55:52,422 DEBUG [StoreOpener-d059c57c2a212eb451f423b021c9a327-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/0bde08697e1442e593d3af4eb18f2f25.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/0bde08697e1442e593d3af4eb18f2f25-bottom 2023-02-09 14:55:52,425 DEBUG [StoreOpener-d059c57c2a212eb451f423b021c9a327-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/1535dc47022d4e169cefe91a535e1ea5.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/1535dc47022d4e169cefe91a535e1ea5-bottom 2023-02-09 14:55:52,428 DEBUG [StoreOpener-d059c57c2a212eb451f423b021c9a327-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/16b9406f9d754f5b8650053c382380fb.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/16b9406f9d754f5b8650053c382380fb-bottom 2023-02-09 14:55:52,431 DEBUG [StoreOpener-d059c57c2a212eb451f423b021c9a327-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/3ddaee433454488594c6ffdf78491e29.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/3ddaee433454488594c6ffdf78491e29-bottom 2023-02-09 14:55:52,434 DEBUG [StoreOpener-d059c57c2a212eb451f423b021c9a327-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/bb3bd5cf3bfb412f8a7efd2ffeefec85.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/bb3bd5cf3bfb412f8a7efd2ffeefec85-bottom 2023-02-09 14:55:52,437 DEBUG [StoreOpener-d059c57c2a212eb451f423b021c9a327-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/e227650000754bd2aabae7f134f5954a.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/e227650000754bd2aabae7f134f5954a-bottom 2023-02-09 14:55:52,440 DEBUG [StoreOpener-d059c57c2a212eb451f423b021c9a327-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/e4c94818f8d2490bb3bac04b76c3da0d.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/e4c94818f8d2490bb3bac04b76c3da0d-bottom 2023-02-09 14:55:52,441 INFO [StoreOpener-d059c57c2a212eb451f423b021c9a327-1] regionserver.HStore(310): Store=d059c57c2a212eb451f423b021c9a327/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:52,441 INFO [StoreOpener-d059c57c2a212eb451f423b021c9a327-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d059c57c2a212eb451f423b021c9a327 2023-02-09 14:55:52,441 DEBUG [StoreOpener-d059c57c2a212eb451f423b021c9a327-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2 2023-02-09 14:55:52,441 DEBUG [StoreOpener-d059c57c2a212eb451f423b021c9a327-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2 2023-02-09 14:55:52,442 INFO [StoreOpener-d059c57c2a212eb451f423b021c9a327-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d059c57c2a212eb451f423b021c9a327 columnFamilyName cf2 2023-02-09 14:55:52,452 DEBUG [StoreOpener-d059c57c2a212eb451f423b021c9a327-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/193b42278c19450c9951535900a995cf.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/193b42278c19450c9951535900a995cf-bottom 2023-02-09 14:55:52,454 DEBUG [StoreOpener-d059c57c2a212eb451f423b021c9a327-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/5bb552d633564dbbac7522449c5dc66f.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/5bb552d633564dbbac7522449c5dc66f-bottom 2023-02-09 14:55:52,457 DEBUG [StoreOpener-d059c57c2a212eb451f423b021c9a327-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/6a12cec5b8ee4a48a36e6ab7d902f86a.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/6a12cec5b8ee4a48a36e6ab7d902f86a-bottom 2023-02-09 14:55:52,460 DEBUG [StoreOpener-d059c57c2a212eb451f423b021c9a327-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/717cc38c7361464593092579c4a3d6ba.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/717cc38c7361464593092579c4a3d6ba-bottom 2023-02-09 14:55:52,463 DEBUG [StoreOpener-d059c57c2a212eb451f423b021c9a327-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/a4c9589bbd744933aa32467915dced43.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/a4c9589bbd744933aa32467915dced43-bottom 2023-02-09 14:55:52,466 DEBUG [StoreOpener-d059c57c2a212eb451f423b021c9a327-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/bc62a2c7a24c49be874e2f455259580e.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/bc62a2c7a24c49be874e2f455259580e-bottom 2023-02-09 14:55:52,470 DEBUG [StoreOpener-d059c57c2a212eb451f423b021c9a327-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/f74cb0ae43544948972923924685b253.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/f74cb0ae43544948972923924685b253-bottom 2023-02-09 14:55:52,470 INFO [StoreOpener-d059c57c2a212eb451f423b021c9a327-1] regionserver.HStore(310): Store=d059c57c2a212eb451f423b021c9a327/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:52,470 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327 2023-02-09 14:55:52,471 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327 2023-02-09 14:55:52,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:55:52,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for d059c57c2a212eb451f423b021c9a327 2023-02-09 14:55:52,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened d059c57c2a212eb451f423b021c9a327; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64473889, jitterRate=-0.03926418721675873}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:55:52,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for d059c57c2a212eb451f423b021c9a327: Running coprocessor pre-open hook at 1675954552409Writing region info on filesystem at 1675954552409Initializing all the Stores at 1675954552409Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954552410 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954552410Cleaning up temporary data from old regions at 1675954552471 (+61 ms)Cleaning up detritus from prior splits at 1675954552472 (+1 ms)Running coprocessor post-open hooks at 1675954552473 (+1 ms)Region opened successfully at 1675954552473 2023-02-09 14:55:52,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327., pid=149, masterSystemTime=1675954552406 2023-02-09 14:55:52,474 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:55:52,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 14:55:52,475 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 14:55:52,475 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 14:55:52,475 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327. 2023-02-09 14:55:52,475 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327. 2023-02-09 14:55:52,475 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HStore(1912): d059c57c2a212eb451f423b021c9a327/cf1 is initiating minor compaction (all files) 2023-02-09 14:55:52,475 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HStore(1912): d059c57c2a212eb451f423b021c9a327/cf2 is initiating minor compaction (all files) 2023-02-09 14:55:52,475 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of d059c57c2a212eb451f423b021c9a327/cf1 in testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327. 2023-02-09 14:55:52,475 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HRegion(2258): Starting compaction of d059c57c2a212eb451f423b021c9a327/cf2 in testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327. 2023-02-09 14:55:52,476 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/717cc38c7361464593092579c4a3d6ba.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/717cc38c7361464593092579c4a3d6ba-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/6a12cec5b8ee4a48a36e6ab7d902f86a.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/6a12cec5b8ee4a48a36e6ab7d902f86a-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/f74cb0ae43544948972923924685b253.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/f74cb0ae43544948972923924685b253-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/a4c9589bbd744933aa32467915dced43.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/a4c9589bbd744933aa32467915dced43-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/bc62a2c7a24c49be874e2f455259580e.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/bc62a2c7a24c49be874e2f455259580e-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/5bb552d633564dbbac7522449c5dc66f.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/5bb552d633564dbbac7522449c5dc66f-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/193b42278c19450c9951535900a995cf.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/193b42278c19450c9951535900a995cf-bottom] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/.tmp, totalSize=34.9 K 2023-02-09 14:55:52,476 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/bb3bd5cf3bfb412f8a7efd2ffeefec85.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/bb3bd5cf3bfb412f8a7efd2ffeefec85-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/e227650000754bd2aabae7f134f5954a.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/e227650000754bd2aabae7f134f5954a-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/1535dc47022d4e169cefe91a535e1ea5.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/1535dc47022d4e169cefe91a535e1ea5-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/e4c94818f8d2490bb3bac04b76c3da0d.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/e4c94818f8d2490bb3bac04b76c3da0d-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/16b9406f9d754f5b8650053c382380fb.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/16b9406f9d754f5b8650053c382380fb-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/3ddaee433454488594c6ffdf78491e29.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/3ddaee433454488594c6ffdf78491e29-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/0bde08697e1442e593d3af4eb18f2f25.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/0bde08697e1442e593d3af4eb18f2f25-bottom] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/.tmp, totalSize=34.9 K 2023-02-09 14:55:52,476 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327. 2023-02-09 14:55:52,476 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting 717cc38c7361464593092579c4a3d6ba.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675954550307 2023-02-09 14:55:52,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327. 2023-02-09 14:55:52,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d. 2023-02-09 14:55:52,476 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting bb3bd5cf3bfb412f8a7efd2ffeefec85.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1675954550307 2023-02-09 14:55:52,476 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 9622b3496718ce3da4db965247f3b82d, NAME => 'testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.', STARTKEY => '41', ENDKEY => ''} 2023-02-09 14:55:52,476 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=d059c57c2a212eb451f423b021c9a327, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:52,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-09 14:55:52,477 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting 6a12cec5b8ee4a48a36e6ab7d902f86a.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675954550472 2023-02-09 14:55:52,477 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675954552476"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954552476"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954552476"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954552476"}]},"ts":"1675954552476"} 2023-02-09 14:55:52,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 9622b3496718ce3da4db965247f3b82d 2023-02-09 14:55:52,477 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting e227650000754bd2aabae7f134f5954a.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1675954550472 2023-02-09 14:55:52,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-09 14:55:52,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 9622b3496718ce3da4db965247f3b82d 2023-02-09 14:55:52,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 9622b3496718ce3da4db965247f3b82d 2023-02-09 14:55:52,477 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting f74cb0ae43544948972923924685b253.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675954550647 2023-02-09 14:55:52,477 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting 1535dc47022d4e169cefe91a535e1ea5.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1675954550647 2023-02-09 14:55:52,478 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting e4c94818f8d2490bb3bac04b76c3da0d.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675954550812 2023-02-09 14:55:52,478 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting a4c9589bbd744933aa32467915dced43.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1675954550812 2023-02-09 14:55:52,478 INFO [StoreOpener-9622b3496718ce3da4db965247f3b82d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 9622b3496718ce3da4db965247f3b82d 2023-02-09 14:55:52,478 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting 16b9406f9d754f5b8650053c382380fb.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675954551002 2023-02-09 14:55:52,478 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting bc62a2c7a24c49be874e2f455259580e.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1675954551002 2023-02-09 14:55:52,479 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting 3ddaee433454488594c6ffdf78491e29.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675954551168 2023-02-09 14:55:52,479 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting 5bb552d633564dbbac7522449c5dc66f.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1675954551168 2023-02-09 14:55:52,479 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting 0bde08697e1442e593d3af4eb18f2f25.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675954551341 2023-02-09 14:55:52,479 DEBUG [StoreOpener-9622b3496718ce3da4db965247f3b82d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1 2023-02-09 14:55:52,479 DEBUG [StoreOpener-9622b3496718ce3da4db965247f3b82d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1 2023-02-09 14:55:52,479 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting 193b42278c19450c9951535900a995cf.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1675954551341 2023-02-09 14:55:52,480 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=147 2023-02-09 14:55:52,480 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=147, state=SUCCESS; OpenRegionProcedure d059c57c2a212eb451f423b021c9a327, server=jenkins-hbase4.apache.org,33423,1675954443133 in 223 msec 2023-02-09 14:55:52,480 INFO [StoreOpener-9622b3496718ce3da4db965247f3b82d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9622b3496718ce3da4db965247f3b82d columnFamilyName cf1 2023-02-09 14:55:52,482 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=d059c57c2a212eb451f423b021c9a327, ASSIGN in 380 msec 2023-02-09 14:55:52,490 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] throttle.PressureAwareThroughputController(145): d059c57c2a212eb451f423b021c9a327#cf1#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:55:52,491 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] throttle.PressureAwareThroughputController(145): d059c57c2a212eb451f423b021c9a327#cf2#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:55:52,492 DEBUG [StoreOpener-9622b3496718ce3da4db965247f3b82d-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/0bde08697e1442e593d3af4eb18f2f25.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/0bde08697e1442e593d3af4eb18f2f25-top 2023-02-09 14:55:52,504 DEBUG [StoreOpener-9622b3496718ce3da4db965247f3b82d-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/1535dc47022d4e169cefe91a535e1ea5.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/1535dc47022d4e169cefe91a535e1ea5-top 2023-02-09 14:55:52,506 DEBUG [StoreOpener-9622b3496718ce3da4db965247f3b82d-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/16b9406f9d754f5b8650053c382380fb.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/16b9406f9d754f5b8650053c382380fb-top 2023-02-09 14:55:52,509 DEBUG [StoreOpener-9622b3496718ce3da4db965247f3b82d-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/3ddaee433454488594c6ffdf78491e29.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/3ddaee433454488594c6ffdf78491e29-top 2023-02-09 14:55:52,520 DEBUG [StoreOpener-9622b3496718ce3da4db965247f3b82d-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/bb3bd5cf3bfb412f8a7efd2ffeefec85.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/bb3bd5cf3bfb412f8a7efd2ffeefec85-top 2023-02-09 14:55:52,523 DEBUG [StoreOpener-9622b3496718ce3da4db965247f3b82d-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/e227650000754bd2aabae7f134f5954a.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/e227650000754bd2aabae7f134f5954a-top 2023-02-09 14:55:52,524 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/.tmp/cf1/ed118671b0af46f08e785e5043e0e01d as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/ed118671b0af46f08e785e5043e0e01d 2023-02-09 14:55:52,525 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/.tmp/cf2/4e701132ae54479eb5e74537f1200b71 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/4e701132ae54479eb5e74537f1200b71 2023-02-09 14:55:52,526 DEBUG [StoreOpener-9622b3496718ce3da4db965247f3b82d-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/e4c94818f8d2490bb3bac04b76c3da0d.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/e4c94818f8d2490bb3bac04b76c3da0d-top 2023-02-09 14:55:52,526 INFO [StoreOpener-9622b3496718ce3da4db965247f3b82d-1] regionserver.HStore(310): Store=9622b3496718ce3da4db965247f3b82d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:52,526 INFO [StoreOpener-9622b3496718ce3da4db965247f3b82d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 9622b3496718ce3da4db965247f3b82d 2023-02-09 14:55:52,527 DEBUG [StoreOpener-9622b3496718ce3da4db965247f3b82d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2 2023-02-09 14:55:52,527 DEBUG [StoreOpener-9622b3496718ce3da4db965247f3b82d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2 2023-02-09 14:55:52,528 INFO [StoreOpener-9622b3496718ce3da4db965247f3b82d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9622b3496718ce3da4db965247f3b82d columnFamilyName cf2 2023-02-09 14:55:52,528 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in d059c57c2a212eb451f423b021c9a327/cf1 of d059c57c2a212eb451f423b021c9a327 into ed118671b0af46f08e785e5043e0e01d(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:55:52,528 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for d059c57c2a212eb451f423b021c9a327: 2023-02-09 14:55:52,528 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327., storeName=d059c57c2a212eb451f423b021c9a327/cf1, priority=9, startTime=1675954552474; duration=0sec 2023-02-09 14:55:52,528 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:55:52,529 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in d059c57c2a212eb451f423b021c9a327/cf2 of d059c57c2a212eb451f423b021c9a327 into 4e701132ae54479eb5e74537f1200b71(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:55:52,529 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for d059c57c2a212eb451f423b021c9a327: 2023-02-09 14:55:52,529 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327., storeName=d059c57c2a212eb451f423b021c9a327/cf2, priority=9, startTime=1675954552475; duration=0sec 2023-02-09 14:55:52,529 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:55:52,537 DEBUG [StoreOpener-9622b3496718ce3da4db965247f3b82d-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/193b42278c19450c9951535900a995cf.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/193b42278c19450c9951535900a995cf-top 2023-02-09 14:55:52,540 DEBUG [StoreOpener-9622b3496718ce3da4db965247f3b82d-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/5bb552d633564dbbac7522449c5dc66f.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/5bb552d633564dbbac7522449c5dc66f-top 2023-02-09 14:55:52,542 DEBUG [StoreOpener-9622b3496718ce3da4db965247f3b82d-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/6a12cec5b8ee4a48a36e6ab7d902f86a.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/6a12cec5b8ee4a48a36e6ab7d902f86a-top 2023-02-09 14:55:52,545 DEBUG [StoreOpener-9622b3496718ce3da4db965247f3b82d-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/717cc38c7361464593092579c4a3d6ba.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/717cc38c7361464593092579c4a3d6ba-top 2023-02-09 14:55:52,548 DEBUG [StoreOpener-9622b3496718ce3da4db965247f3b82d-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/a4c9589bbd744933aa32467915dced43.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/a4c9589bbd744933aa32467915dced43-top 2023-02-09 14:55:52,550 DEBUG [StoreOpener-9622b3496718ce3da4db965247f3b82d-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/bc62a2c7a24c49be874e2f455259580e.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/bc62a2c7a24c49be874e2f455259580e-top 2023-02-09 14:55:52,553 DEBUG [StoreOpener-9622b3496718ce3da4db965247f3b82d-1] regionserver.HStore(539): loaded hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/f74cb0ae43544948972923924685b253.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/f74cb0ae43544948972923924685b253-top 2023-02-09 14:55:52,553 INFO [StoreOpener-9622b3496718ce3da4db965247f3b82d-1] regionserver.HStore(310): Store=9622b3496718ce3da4db965247f3b82d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-09 14:55:52,554 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d 2023-02-09 14:55:52,554 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d 2023-02-09 14:55:52,555 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-09 14:55:52,556 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 9622b3496718ce3da4db965247f3b82d 2023-02-09 14:55:52,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 9622b3496718ce3da4db965247f3b82d; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65766415, jitterRate=-0.020004048943519592}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-09 14:55:52,557 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 9622b3496718ce3da4db965247f3b82d: Running coprocessor pre-open hook at 1675954552477Writing region info on filesystem at 1675954552477Initializing all the Stores at 1675954552478 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954552478Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675954552478Cleaning up temporary data from old regions at 1675954552555 (+77 ms)Cleaning up detritus from prior splits at 1675954552555Running coprocessor post-open hooks at 1675954552557 (+2 ms)Region opened successfully at 1675954552557 2023-02-09 14:55:52,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d., pid=150, masterSystemTime=1675954552406 2023-02-09 14:55:52,557 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:55:52,558 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 14:55:52,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-09 14:55:52,559 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-09 14:55:52,559 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d. 2023-02-09 14:55:52,559 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HStore(1912): 9622b3496718ce3da4db965247f3b82d/cf1 is initiating minor compaction (all files) 2023-02-09 14:55:52,559 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d. 2023-02-09 14:55:52,559 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HStore(1912): 9622b3496718ce3da4db965247f3b82d/cf2 is initiating minor compaction (all files) 2023-02-09 14:55:52,559 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 9622b3496718ce3da4db965247f3b82d/cf1 in testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d. 2023-02-09 14:55:52,559 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 9622b3496718ce3da4db965247f3b82d/cf2 in testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d. 2023-02-09 14:55:52,559 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/bb3bd5cf3bfb412f8a7efd2ffeefec85.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/bb3bd5cf3bfb412f8a7efd2ffeefec85-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/e227650000754bd2aabae7f134f5954a.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/e227650000754bd2aabae7f134f5954a-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/1535dc47022d4e169cefe91a535e1ea5.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/1535dc47022d4e169cefe91a535e1ea5-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/e4c94818f8d2490bb3bac04b76c3da0d.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/e4c94818f8d2490bb3bac04b76c3da0d-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/16b9406f9d754f5b8650053c382380fb.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/16b9406f9d754f5b8650053c382380fb-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/3ddaee433454488594c6ffdf78491e29.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/3ddaee433454488594c6ffdf78491e29-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/0bde08697e1442e593d3af4eb18f2f25.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/0bde08697e1442e593d3af4eb18f2f25-top] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/.tmp, totalSize=34.9 K 2023-02-09 14:55:52,559 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/717cc38c7361464593092579c4a3d6ba.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/717cc38c7361464593092579c4a3d6ba-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/6a12cec5b8ee4a48a36e6ab7d902f86a.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/6a12cec5b8ee4a48a36e6ab7d902f86a-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/f74cb0ae43544948972923924685b253.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/f74cb0ae43544948972923924685b253-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/a4c9589bbd744933aa32467915dced43.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/a4c9589bbd744933aa32467915dced43-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/bc62a2c7a24c49be874e2f455259580e.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/bc62a2c7a24c49be874e2f455259580e-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/5bb552d633564dbbac7522449c5dc66f.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/5bb552d633564dbbac7522449c5dc66f-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/193b42278c19450c9951535900a995cf.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/193b42278c19450c9951535900a995cf-top] into tmpdir=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/.tmp, totalSize=34.9 K 2023-02-09 14:55:52,560 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting bb3bd5cf3bfb412f8a7efd2ffeefec85.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675954550307 2023-02-09 14:55:52,560 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting 717cc38c7361464593092579c4a3d6ba.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1675954550307 2023-02-09 14:55:52,560 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d. 2023-02-09 14:55:52,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d. 2023-02-09 14:55:52,560 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting e227650000754bd2aabae7f134f5954a.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675954550472 2023-02-09 14:55:52,560 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=9622b3496718ce3da4db965247f3b82d, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:52,560 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting 6a12cec5b8ee4a48a36e6ab7d902f86a.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1675954550472 2023-02-09 14:55:52,560 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675954552560"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675954552560"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675954552560"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675954552560"}]},"ts":"1675954552560"} 2023-02-09 14:55:52,561 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting 1535dc47022d4e169cefe91a535e1ea5.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675954550647 2023-02-09 14:55:52,561 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting f74cb0ae43544948972923924685b253.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1675954550647 2023-02-09 14:55:52,561 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting e4c94818f8d2490bb3bac04b76c3da0d.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675954550812 2023-02-09 14:55:52,561 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting a4c9589bbd744933aa32467915dced43.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1675954550812 2023-02-09 14:55:52,562 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting 16b9406f9d754f5b8650053c382380fb.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675954551002 2023-02-09 14:55:52,562 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting bc62a2c7a24c49be874e2f455259580e.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1675954551002 2023-02-09 14:55:52,562 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting 5bb552d633564dbbac7522449c5dc66f.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675954551168 2023-02-09 14:55:52,562 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting 3ddaee433454488594c6ffdf78491e29.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1675954551168 2023-02-09 14:55:52,562 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] compactions.Compactor(207): Compacting 193b42278c19450c9951535900a995cf.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675954551341 2023-02-09 14:55:52,563 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] compactions.Compactor(207): Compacting 0bde08697e1442e593d3af4eb18f2f25.2a77ee9910df938efcf9f9f8fd3a86cb, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1675954551341 2023-02-09 14:55:52,564 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=148 2023-02-09 14:55:52,564 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=148, state=SUCCESS; OpenRegionProcedure 9622b3496718ce3da4db965247f3b82d, server=jenkins-hbase4.apache.org,33423,1675954443133 in 304 msec 2023-02-09 14:55:52,565 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=148, resume processing ppid=144 2023-02-09 14:55:52,565 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=9622b3496718ce3da4db965247f3b82d, ASSIGN in 464 msec 2023-02-09 14:55:52,566 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=2a77ee9910df938efcf9f9f8fd3a86cb, daughterA=d059c57c2a212eb451f423b021c9a327, daughterB=9622b3496718ce3da4db965247f3b82d in 1.2180 sec 2023-02-09 14:55:52,572 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] throttle.PressureAwareThroughputController(145): 9622b3496718ce3da4db965247f3b82d#cf2#compaction#144 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:55:52,573 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] throttle.PressureAwareThroughputController(145): 9622b3496718ce3da4db965247f3b82d#cf1#compaction#145 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-09 14:55:52,588 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/.tmp/cf2/b2c4df98d1574856bbb48a2cb47a7579 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/b2c4df98d1574856bbb48a2cb47a7579 2023-02-09 14:55:52,589 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/.tmp/cf1/7824e22a60c14e2194ba4c71bafd7933 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/7824e22a60c14e2194ba4c71bafd7933 2023-02-09 14:55:52,592 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 9622b3496718ce3da4db965247f3b82d/cf2 of 9622b3496718ce3da4db965247f3b82d into b2c4df98d1574856bbb48a2cb47a7579(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:55:52,592 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 9622b3496718ce3da4db965247f3b82d: 2023-02-09 14:55:52,592 INFO [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d., storeName=9622b3496718ce3da4db965247f3b82d/cf2, priority=9, startTime=1675954552558; duration=0sec 2023-02-09 14:55:52,592 DEBUG [RS:0;jenkins-hbase4:33423-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:55:52,594 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 9622b3496718ce3da4db965247f3b82d/cf1 of 9622b3496718ce3da4db965247f3b82d into 7824e22a60c14e2194ba4c71bafd7933(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-09 14:55:52,594 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 9622b3496718ce3da4db965247f3b82d: 2023-02-09 14:55:52,594 INFO [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d., storeName=9622b3496718ce3da4db965247f3b82d/cf1, priority=9, startTime=1675954552557; duration=0sec 2023-02-09 14:55:52,594 DEBUG [RS:0;jenkins-hbase4:33423-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-09 14:55:52,599 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33423] ipc.CallRunner(144): callId: 1116 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:47026 deadline: 1675954612598, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb. is not online on jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:52,709 DEBUG [Listener at localhost/40823] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 14:55:52,713 INFO [Listener at localhost/40823] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-09 14:55:52,715 DEBUG [Listener at localhost/40823] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-09 14:55:52,723 WARN [Listener at localhost/40823] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-09 14:55:52,723 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:52,724 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:52,725 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:52,725 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:52,725 INFO [Listener at localhost/40823] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-02-09 14:55:52,725 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testSplitTableRegion 2023-02-09 14:55:52,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-02-09 14:55:52,728 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:52,728 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:52,728 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954552728"}]},"ts":"1675954552728"} 2023-02-09 14:55:52,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-09 14:55:52,730 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-02-09 14:55:52,731 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-02-09 14:55:52,732 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=d059c57c2a212eb451f423b021c9a327, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=9622b3496718ce3da4db965247f3b82d, UNASSIGN}] 2023-02-09 14:55:52,732 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=9622b3496718ce3da4db965247f3b82d, UNASSIGN 2023-02-09 14:55:52,733 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=9622b3496718ce3da4db965247f3b82d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:52,733 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675954552733"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954552733"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954552733"}]},"ts":"1675954552733"} 2023-02-09 14:55:52,734 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 9622b3496718ce3da4db965247f3b82d, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:55:52,736 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=d059c57c2a212eb451f423b021c9a327, UNASSIGN 2023-02-09 14:55:52,736 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=d059c57c2a212eb451f423b021c9a327, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:52,736 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675954552736"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675954552736"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675954552736"}]},"ts":"1675954552736"} 2023-02-09 14:55:52,739 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure d059c57c2a212eb451f423b021c9a327, server=jenkins-hbase4.apache.org,33423,1675954443133}] 2023-02-09 14:55:52,829 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:52,829 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:52,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-09 14:55:52,885 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 9622b3496718ce3da4db965247f3b82d 2023-02-09 14:55:52,886 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 9622b3496718ce3da4db965247f3b82d, disabling compactions & flushes 2023-02-09 14:55:52,886 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d. 2023-02-09 14:55:52,886 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d. 2023-02-09 14:55:52,886 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d. after waiting 0 ms 2023-02-09 14:55:52,886 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d. 2023-02-09 14:55:52,886 DEBUG [StoreCloser-testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/bb3bd5cf3bfb412f8a7efd2ffeefec85.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/bb3bd5cf3bfb412f8a7efd2ffeefec85-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/e227650000754bd2aabae7f134f5954a.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/e227650000754bd2aabae7f134f5954a-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/1535dc47022d4e169cefe91a535e1ea5.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/1535dc47022d4e169cefe91a535e1ea5-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/e4c94818f8d2490bb3bac04b76c3da0d.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/e4c94818f8d2490bb3bac04b76c3da0d-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/16b9406f9d754f5b8650053c382380fb.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/16b9406f9d754f5b8650053c382380fb-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/3ddaee433454488594c6ffdf78491e29.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/3ddaee433454488594c6ffdf78491e29-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/0bde08697e1442e593d3af4eb18f2f25.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/0bde08697e1442e593d3af4eb18f2f25-top] to archive 2023-02-09 14:55:52,887 DEBUG [StoreCloser-testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:52,888 DEBUG [StoreCloser-testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/bb3bd5cf3bfb412f8a7efd2ffeefec85.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/bb3bd5cf3bfb412f8a7efd2ffeefec85.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,889 DEBUG [StoreCloser-testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/e227650000754bd2aabae7f134f5954a.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/e227650000754bd2aabae7f134f5954a.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,890 DEBUG [StoreCloser-testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/1535dc47022d4e169cefe91a535e1ea5.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/1535dc47022d4e169cefe91a535e1ea5.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,891 DEBUG [StoreCloser-testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/e4c94818f8d2490bb3bac04b76c3da0d.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/e4c94818f8d2490bb3bac04b76c3da0d.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,892 DEBUG [StoreCloser-testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/16b9406f9d754f5b8650053c382380fb.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/16b9406f9d754f5b8650053c382380fb.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,920 DEBUG [StoreCloser-testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/3ddaee433454488594c6ffdf78491e29.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/3ddaee433454488594c6ffdf78491e29.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,921 DEBUG [StoreCloser-testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/0bde08697e1442e593d3af4eb18f2f25.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/0bde08697e1442e593d3af4eb18f2f25.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,922 DEBUG [StoreCloser-testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/717cc38c7361464593092579c4a3d6ba.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/717cc38c7361464593092579c4a3d6ba-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/6a12cec5b8ee4a48a36e6ab7d902f86a.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/6a12cec5b8ee4a48a36e6ab7d902f86a-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/f74cb0ae43544948972923924685b253.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/f74cb0ae43544948972923924685b253-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/a4c9589bbd744933aa32467915dced43.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/a4c9589bbd744933aa32467915dced43-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/bc62a2c7a24c49be874e2f455259580e.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/bc62a2c7a24c49be874e2f455259580e-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/5bb552d633564dbbac7522449c5dc66f.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/5bb552d633564dbbac7522449c5dc66f-top, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/193b42278c19450c9951535900a995cf.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/193b42278c19450c9951535900a995cf-top] to archive 2023-02-09 14:55:52,922 DEBUG [StoreCloser-testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:52,924 DEBUG [StoreCloser-testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/717cc38c7361464593092579c4a3d6ba.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/717cc38c7361464593092579c4a3d6ba.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,925 DEBUG [StoreCloser-testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/6a12cec5b8ee4a48a36e6ab7d902f86a.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/6a12cec5b8ee4a48a36e6ab7d902f86a.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,926 DEBUG [StoreCloser-testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/f74cb0ae43544948972923924685b253.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/f74cb0ae43544948972923924685b253.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,927 DEBUG [StoreCloser-testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/a4c9589bbd744933aa32467915dced43.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/a4c9589bbd744933aa32467915dced43.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,928 DEBUG [StoreCloser-testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/bc62a2c7a24c49be874e2f455259580e.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/bc62a2c7a24c49be874e2f455259580e.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,929 DEBUG [StoreCloser-testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/5bb552d633564dbbac7522449c5dc66f.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/5bb552d633564dbbac7522449c5dc66f.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,930 DEBUG [StoreCloser-testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/193b42278c19450c9951535900a995cf.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/193b42278c19450c9951535900a995cf.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,932 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 14:55:52,933 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:55:52,933 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d. 2023-02-09 14:55:52,933 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 9622b3496718ce3da4db965247f3b82d: Waiting for close lock at 1675954552886Running coprocessor pre-close hooks at 1675954552886Disabling compacts and flushes for region at 1675954552886Disabling writes for close at 1675954552886Writing region close event to WAL at 1675954552930 (+44 ms)Running coprocessor post-close hooks at 1675954552933 (+3 ms)Closed at 1675954552933 2023-02-09 14:55:52,934 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 9622b3496718ce3da4db965247f3b82d 2023-02-09 14:55:52,934 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close d059c57c2a212eb451f423b021c9a327 2023-02-09 14:55:52,934 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing d059c57c2a212eb451f423b021c9a327, disabling compactions & flushes 2023-02-09 14:55:52,934 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327. 2023-02-09 14:55:52,934 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327. 2023-02-09 14:55:52,934 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327. after waiting 0 ms 2023-02-09 14:55:52,934 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327. 2023-02-09 14:55:52,934 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=9622b3496718ce3da4db965247f3b82d, regionState=CLOSED 2023-02-09 14:55:52,934 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675954552934"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954552934"}]},"ts":"1675954552934"} 2023-02-09 14:55:52,935 DEBUG [StoreCloser-testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/bb3bd5cf3bfb412f8a7efd2ffeefec85.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/bb3bd5cf3bfb412f8a7efd2ffeefec85-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/e227650000754bd2aabae7f134f5954a.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/e227650000754bd2aabae7f134f5954a-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/1535dc47022d4e169cefe91a535e1ea5.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/1535dc47022d4e169cefe91a535e1ea5-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/e4c94818f8d2490bb3bac04b76c3da0d.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/e4c94818f8d2490bb3bac04b76c3da0d-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/16b9406f9d754f5b8650053c382380fb.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/16b9406f9d754f5b8650053c382380fb-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/3ddaee433454488594c6ffdf78491e29.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/3ddaee433454488594c6ffdf78491e29-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/0bde08697e1442e593d3af4eb18f2f25.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/0bde08697e1442e593d3af4eb18f2f25-bottom] to archive 2023-02-09 14:55:52,936 DEBUG [StoreCloser-testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:52,937 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-02-09 14:55:52,937 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 9622b3496718ce3da4db965247f3b82d, server=jenkins-hbase4.apache.org,33423,1675954443133 in 201 msec 2023-02-09 14:55:52,937 DEBUG [StoreCloser-testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/bb3bd5cf3bfb412f8a7efd2ffeefec85.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/bb3bd5cf3bfb412f8a7efd2ffeefec85.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,938 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=9622b3496718ce3da4db965247f3b82d, UNASSIGN in 205 msec 2023-02-09 14:55:52,938 DEBUG [StoreCloser-testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/e227650000754bd2aabae7f134f5954a.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/e227650000754bd2aabae7f134f5954a.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,939 DEBUG [StoreCloser-testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/1535dc47022d4e169cefe91a535e1ea5.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/1535dc47022d4e169cefe91a535e1ea5.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,940 DEBUG [StoreCloser-testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/e4c94818f8d2490bb3bac04b76c3da0d.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/e4c94818f8d2490bb3bac04b76c3da0d.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,940 DEBUG [StoreCloser-testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/16b9406f9d754f5b8650053c382380fb.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/16b9406f9d754f5b8650053c382380fb.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,941 DEBUG [StoreCloser-testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/3ddaee433454488594c6ffdf78491e29.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/3ddaee433454488594c6ffdf78491e29.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,942 DEBUG [StoreCloser-testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/0bde08697e1442e593d3af4eb18f2f25.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/0bde08697e1442e593d3af4eb18f2f25.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,943 DEBUG [StoreCloser-testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/717cc38c7361464593092579c4a3d6ba.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/717cc38c7361464593092579c4a3d6ba-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/6a12cec5b8ee4a48a36e6ab7d902f86a.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/6a12cec5b8ee4a48a36e6ab7d902f86a-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/f74cb0ae43544948972923924685b253.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/f74cb0ae43544948972923924685b253-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/a4c9589bbd744933aa32467915dced43.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/a4c9589bbd744933aa32467915dced43-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/bc62a2c7a24c49be874e2f455259580e.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/bc62a2c7a24c49be874e2f455259580e-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/5bb552d633564dbbac7522449c5dc66f.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/5bb552d633564dbbac7522449c5dc66f-bottom, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/193b42278c19450c9951535900a995cf.2a77ee9910df938efcf9f9f8fd3a86cb->hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/193b42278c19450c9951535900a995cf-bottom] to archive 2023-02-09 14:55:52,944 DEBUG [StoreCloser-testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-09 14:55:52,945 DEBUG [StoreCloser-testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/717cc38c7361464593092579c4a3d6ba.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/717cc38c7361464593092579c4a3d6ba.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,946 DEBUG [StoreCloser-testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/6a12cec5b8ee4a48a36e6ab7d902f86a.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/6a12cec5b8ee4a48a36e6ab7d902f86a.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,947 DEBUG [StoreCloser-testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/f74cb0ae43544948972923924685b253.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/f74cb0ae43544948972923924685b253.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,947 DEBUG [StoreCloser-testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/a4c9589bbd744933aa32467915dced43.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/a4c9589bbd744933aa32467915dced43.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,948 DEBUG [StoreCloser-testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/bc62a2c7a24c49be874e2f455259580e.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/bc62a2c7a24c49be874e2f455259580e.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,949 DEBUG [StoreCloser-testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/5bb552d633564dbbac7522449c5dc66f.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/5bb552d633564dbbac7522449c5dc66f.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,950 DEBUG [StoreCloser-testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/193b42278c19450c9951535900a995cf.2a77ee9910df938efcf9f9f8fd3a86cb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/193b42278c19450c9951535900a995cf.2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:52,952 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-09 14:55:52,953 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:55:52,953 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327. 2023-02-09 14:55:52,953 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for d059c57c2a212eb451f423b021c9a327: Waiting for close lock at 1675954552934Running coprocessor pre-close hooks at 1675954552934Disabling compacts and flushes for region at 1675954552934Disabling writes for close at 1675954552934Writing region close event to WAL at 1675954552950 (+16 ms)Running coprocessor post-close hooks at 1675954552953 (+3 ms)Closed at 1675954552953 2023-02-09 14:55:52,954 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed d059c57c2a212eb451f423b021c9a327 2023-02-09 14:55:52,954 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=d059c57c2a212eb451f423b021c9a327, regionState=CLOSED 2023-02-09 14:55:52,954 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675954552954"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675954552954"}]},"ts":"1675954552954"} 2023-02-09 14:55:52,956 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-02-09 14:55:52,956 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure d059c57c2a212eb451f423b021c9a327, server=jenkins-hbase4.apache.org,33423,1675954443133 in 218 msec 2023-02-09 14:55:52,957 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=152, resume processing ppid=151 2023-02-09 14:55:52,957 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=d059c57c2a212eb451f423b021c9a327, UNASSIGN in 224 msec 2023-02-09 14:55:52,958 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675954552958"}]},"ts":"1675954552958"} 2023-02-09 14:55:52,959 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-02-09 14:55:52,960 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-02-09 14:55:52,961 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 235 msec 2023-02-09 14:55:53,030 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:53,030 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:53,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-09 14:55:53,031 INFO [Listener at localhost/40823] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-02-09 14:55:53,031 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:53,031 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:53,032 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testSplitTableRegion 2023-02-09 14:55:53,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-02-09 14:55:53,034 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-09 14:55:53,034 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:53,034 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:53,035 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-09 14:55:53,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-09 14:55:53,038 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:53,038 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d 2023-02-09 14:55:53,038 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327 2023-02-09 14:55:53,040 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/recovered.edits] 2023-02-09 14:55:53,040 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/recovered.edits] 2023-02-09 14:55:53,040 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2, FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/recovered.edits] 2023-02-09 14:55:53,043 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/ed118671b0af46f08e785e5043e0e01d to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf1/ed118671b0af46f08e785e5043e0e01d 2023-02-09 14:55:53,043 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/7824e22a60c14e2194ba4c71bafd7933 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf1/7824e22a60c14e2194ba4c71bafd7933 2023-02-09 14:55:53,043 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/0bde08697e1442e593d3af4eb18f2f25 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/0bde08697e1442e593d3af4eb18f2f25 2023-02-09 14:55:53,045 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/1535dc47022d4e169cefe91a535e1ea5 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/1535dc47022d4e169cefe91a535e1ea5 2023-02-09 14:55:53,045 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/4e701132ae54479eb5e74537f1200b71 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/cf2/4e701132ae54479eb5e74537f1200b71 2023-02-09 14:55:53,045 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/b2c4df98d1574856bbb48a2cb47a7579 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/cf2/b2c4df98d1574856bbb48a2cb47a7579 2023-02-09 14:55:53,046 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/16b9406f9d754f5b8650053c382380fb to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/16b9406f9d754f5b8650053c382380fb 2023-02-09 14:55:53,047 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/3ddaee433454488594c6ffdf78491e29 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/3ddaee433454488594c6ffdf78491e29 2023-02-09 14:55:53,047 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/recovered.edits/90.seqid to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327/recovered.edits/90.seqid 2023-02-09 14:55:53,048 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/recovered.edits/90.seqid to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d/recovered.edits/90.seqid 2023-02-09 14:55:53,048 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/d059c57c2a212eb451f423b021c9a327 2023-02-09 14:55:53,048 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/bb3bd5cf3bfb412f8a7efd2ffeefec85 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/bb3bd5cf3bfb412f8a7efd2ffeefec85 2023-02-09 14:55:53,048 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/9622b3496718ce3da4db965247f3b82d 2023-02-09 14:55:53,049 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/e227650000754bd2aabae7f134f5954a to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/e227650000754bd2aabae7f134f5954a 2023-02-09 14:55:53,050 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/e4c94818f8d2490bb3bac04b76c3da0d to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf1/e4c94818f8d2490bb3bac04b76c3da0d 2023-02-09 14:55:53,052 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/193b42278c19450c9951535900a995cf to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/193b42278c19450c9951535900a995cf 2023-02-09 14:55:53,053 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/5bb552d633564dbbac7522449c5dc66f to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/5bb552d633564dbbac7522449c5dc66f 2023-02-09 14:55:53,053 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/6a12cec5b8ee4a48a36e6ab7d902f86a to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/6a12cec5b8ee4a48a36e6ab7d902f86a 2023-02-09 14:55:53,054 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/717cc38c7361464593092579c4a3d6ba to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/717cc38c7361464593092579c4a3d6ba 2023-02-09 14:55:53,055 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/a4c9589bbd744933aa32467915dced43 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/a4c9589bbd744933aa32467915dced43 2023-02-09 14:55:53,056 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/bc62a2c7a24c49be874e2f455259580e to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/bc62a2c7a24c49be874e2f455259580e 2023-02-09 14:55:53,057 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/f74cb0ae43544948972923924685b253 to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/cf2/f74cb0ae43544948972923924685b253 2023-02-09 14:55:53,058 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/recovered.edits/85.seqid to hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/archive/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb/recovered.edits/85.seqid 2023-02-09 14:55:53,059 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/.tmp/data/default/testSplitTableRegion/2a77ee9910df938efcf9f9f8fd3a86cb 2023-02-09 14:55:53,059 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-09 14:55:53,060 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-09 14:55:53,062 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-02-09 14:55:53,063 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-02-09 14:55:53,064 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-09 14:55:53,064 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-02-09 14:55:53,064 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954553064"}]},"ts":"9223372036854775807"} 2023-02-09 14:55:53,064 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954553064"}]},"ts":"9223372036854775807"} 2023-02-09 14:55:53,064 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675954553064"}]},"ts":"9223372036854775807"} 2023-02-09 14:55:53,065 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-09 14:55:53,065 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 2a77ee9910df938efcf9f9f8fd3a86cb, NAME => 'testSplitTableRegion,,1675954549804.2a77ee9910df938efcf9f9f8fd3a86cb.', STARTKEY => '', ENDKEY => ''}, {ENCODED => d059c57c2a212eb451f423b021c9a327, NAME => 'testSplitTableRegion,,1675954551347.d059c57c2a212eb451f423b021c9a327.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 9622b3496718ce3da4db965247f3b82d, NAME => 'testSplitTableRegion,41,1675954551347.9622b3496718ce3da4db965247f3b82d.', STARTKEY => '41', ENDKEY => ''}] 2023-02-09 14:55:53,065 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-02-09 14:55:53,066 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675954553065"}]},"ts":"9223372036854775807"} 2023-02-09 14:55:53,066 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-02-09 14:55:53,068 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-09 14:55:53,069 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 36 msec 2023-02-09 14:55:53,135 INFO [Listener at localhost/40823] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-09 14:55:53,135 INFO [Listener at localhost/40823] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-09 14:55:53,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=43829] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-09 14:55:53,136 INFO [Listener at localhost/40823] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-02-09 14:55:53,150 INFO [Listener at localhost/40823] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=377 (was 374) - Thread LEAK? -, OpenFileDescriptor=815 (was 833), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=86 (was 85) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=2887 (was 2941) 2023-02-09 14:55:53,150 INFO [Listener at localhost/40823] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-09 14:55:53,150 INFO [Listener at localhost/40823] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-09 14:55:53,153 DEBUG [Listener at localhost/40823] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x37e887de to 127.0.0.1:52509 2023-02-09 14:55:53,154 DEBUG [Listener at localhost/40823] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:55:53,154 DEBUG [Listener at localhost/40823] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-09 14:55:53,155 DEBUG [Listener at localhost/40823] util.JVMClusterUtil(257): Found active master hash=1943575103, stopped=false 2023-02-09 14:55:53,155 INFO [Listener at localhost/40823] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,43829,1675954441993 2023-02-09 14:55:53,156 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-09 14:55:53,156 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-09 14:55:53,156 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-09 14:55:53,156 INFO [Listener at localhost/40823] procedure2.ProcedureExecutor(629): Stopping 2023-02-09 14:55:53,156 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-09 14:55:53,157 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 14:55:53,157 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 14:55:53,157 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 14:55:53,157 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 14:55:53,157 DEBUG [Listener at localhost/40823] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x555f3efe to 127.0.0.1:52509 2023-02-09 14:55:53,157 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-09 14:55:53,157 DEBUG [Listener at localhost/40823] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:55:53,157 INFO [Listener at localhost/40823] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,33423,1675954443133' ***** 2023-02-09 14:55:53,158 INFO [Listener at localhost/40823] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-09 14:55:53,158 INFO [Listener at localhost/40823] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,33553,1675954443180' ***** 2023-02-09 14:55:53,158 INFO [Listener at localhost/40823] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-09 14:55:53,158 INFO [Listener at localhost/40823] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,41077,1675954443212' ***** 2023-02-09 14:55:53,158 INFO [Listener at localhost/40823] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-09 14:55:53,158 INFO [RS:2;jenkins-hbase4:41077] regionserver.HeapMemoryManager(220): Stopping 2023-02-09 14:55:53,158 INFO [RS:0;jenkins-hbase4:33423] regionserver.HeapMemoryManager(220): Stopping 2023-02-09 14:55:53,158 INFO [RS:1;jenkins-hbase4:33553] regionserver.HeapMemoryManager(220): Stopping 2023-02-09 14:55:53,158 INFO [RS:2;jenkins-hbase4:41077] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-09 14:55:53,158 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-09 14:55:53,158 INFO [RS:1;jenkins-hbase4:33553] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-09 14:55:53,158 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-09 14:55:53,158 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-09 14:55:53,158 INFO [RS:1;jenkins-hbase4:33553] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-09 14:55:53,158 INFO [RS:0;jenkins-hbase4:33423] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-09 14:55:53,159 INFO [RS:1;jenkins-hbase4:33553] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:53,159 INFO [RS:0;jenkins-hbase4:33423] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-09 14:55:53,159 DEBUG [RS:1;jenkins-hbase4:33553] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x49fba862 to 127.0.0.1:52509 2023-02-09 14:55:53,158 INFO [RS:2;jenkins-hbase4:41077] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-09 14:55:53,159 INFO [RS:0;jenkins-hbase4:33423] regionserver.HRegionServer(3304): Received CLOSE for 0c1f8115bde7013387f9461b40485fe1 2023-02-09 14:55:53,159 INFO [RS:2;jenkins-hbase4:41077] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:53,159 DEBUG [RS:1;jenkins-hbase4:33553] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:55:53,159 DEBUG [RS:2;jenkins-hbase4:41077] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x749237b6 to 127.0.0.1:52509 2023-02-09 14:55:53,159 INFO [RS:1;jenkins-hbase4:33553] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,33553,1675954443180; all regions closed. 2023-02-09 14:55:53,159 DEBUG [RS:2;jenkins-hbase4:41077] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:55:53,159 INFO [RS:0;jenkins-hbase4:33423] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:53,159 INFO [RS:2;jenkins-hbase4:41077] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,41077,1675954443212; all regions closed. 2023-02-09 14:55:53,159 DEBUG [RS:0;jenkins-hbase4:33423] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x25af0fff to 127.0.0.1:52509 2023-02-09 14:55:53,159 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 0c1f8115bde7013387f9461b40485fe1, disabling compactions & flushes 2023-02-09 14:55:53,159 DEBUG [RS:0;jenkins-hbase4:33423] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:55:53,159 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675954445639.0c1f8115bde7013387f9461b40485fe1. 2023-02-09 14:55:53,159 INFO [RS:0;jenkins-hbase4:33423] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-09 14:55:53,159 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675954445639.0c1f8115bde7013387f9461b40485fe1. 2023-02-09 14:55:53,159 INFO [RS:0;jenkins-hbase4:33423] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-09 14:55:53,160 INFO [RS:0;jenkins-hbase4:33423] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-09 14:55:53,160 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675954445639.0c1f8115bde7013387f9461b40485fe1. after waiting 0 ms 2023-02-09 14:55:53,160 INFO [RS:0;jenkins-hbase4:33423] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-09 14:55:53,160 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675954445639.0c1f8115bde7013387f9461b40485fe1. 2023-02-09 14:55:53,160 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 0c1f8115bde7013387f9461b40485fe1 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-09 14:55:53,160 INFO [RS:0;jenkins-hbase4:33423] regionserver.HRegionServer(1475): Waiting on 2 regions to close 2023-02-09 14:55:53,160 DEBUG [RS:0;jenkins-hbase4:33423] regionserver.HRegionServer(1479): Online Regions={0c1f8115bde7013387f9461b40485fe1=hbase:namespace,,1675954445639.0c1f8115bde7013387f9461b40485fe1., 1588230740=hbase:meta,,1.1588230740} 2023-02-09 14:55:53,160 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-09 14:55:53,160 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-09 14:55:53,160 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-09 14:55:53,160 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-09 14:55:53,160 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-09 14:55:53,160 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=79.40 KB heapSize=125.77 KB 2023-02-09 14:55:53,161 DEBUG [RS:0;jenkins-hbase4:33423] regionserver.HRegionServer(1505): Waiting on 0c1f8115bde7013387f9461b40485fe1, 1588230740 2023-02-09 14:55:53,170 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-09 14:55:53,176 DEBUG [RS:1;jenkins-hbase4:33553] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/oldWALs 2023-02-09 14:55:53,177 INFO [RS:1;jenkins-hbase4:33553] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33553%2C1675954443180:(num 1675954444846) 2023-02-09 14:55:53,177 DEBUG [RS:1;jenkins-hbase4:33553] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:55:53,177 INFO [RS:1;jenkins-hbase4:33553] regionserver.LeaseManager(133): Closed leases 2023-02-09 14:55:53,177 DEBUG [RS:2;jenkins-hbase4:41077] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/oldWALs 2023-02-09 14:55:53,177 INFO [RS:2;jenkins-hbase4:41077] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41077%2C1675954443212:(num 1675954444843) 2023-02-09 14:55:53,177 DEBUG [RS:2;jenkins-hbase4:41077] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:55:53,177 INFO [RS:1;jenkins-hbase4:33553] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-09 14:55:53,177 INFO [RS:2;jenkins-hbase4:41077] regionserver.LeaseManager(133): Closed leases 2023-02-09 14:55:53,177 INFO [RS:1;jenkins-hbase4:33553] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-09 14:55:53,177 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 14:55:53,177 INFO [RS:1;jenkins-hbase4:33553] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-09 14:55:53,178 INFO [RS:1;jenkins-hbase4:33553] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-09 14:55:53,178 INFO [RS:2;jenkins-hbase4:41077] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-09 14:55:53,179 INFO [RS:2;jenkins-hbase4:41077] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-09 14:55:53,179 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 14:55:53,179 INFO [RS:2;jenkins-hbase4:41077] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-09 14:55:53,179 INFO [RS:2;jenkins-hbase4:41077] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-09 14:55:53,179 INFO [RS:2;jenkins-hbase4:41077] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41077 2023-02-09 14:55:53,180 INFO [RS:1;jenkins-hbase4:33553] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33553 2023-02-09 14:55:53,183 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.29 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740/.tmp/info/e9409d35ca234c7eb762af27e5bb4691 2023-02-09 14:55:53,183 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/namespace/0c1f8115bde7013387f9461b40485fe1/.tmp/info/f8cd2ab33b4c4963b5452f2b9f3808ef 2023-02-09 14:55:53,339 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-09 14:55:53,339 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-09 14:55:53,344 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:53,344 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 14:55:53,344 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e9409d35ca234c7eb762af27e5bb4691 2023-02-09 14:55:53,344 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 14:55:53,344 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:53,344 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33553,1675954443180 2023-02-09 14:55:53,344 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:53,344 ERROR [Listener at localhost/40823-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3845ae85 rejected from java.util.concurrent.ThreadPoolExecutor@3c7a2d55[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 14:55:53,345 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/namespace/0c1f8115bde7013387f9461b40485fe1/.tmp/info/f8cd2ab33b4c4963b5452f2b9f3808ef as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/namespace/0c1f8115bde7013387f9461b40485fe1/info/f8cd2ab33b4c4963b5452f2b9f3808ef 2023-02-09 14:55:53,345 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 14:55:53,344 ERROR [Listener at localhost/40823-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2fefa346 rejected from java.util.concurrent.ThreadPoolExecutor@2bd02263[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 14:55:53,345 ERROR [Listener at localhost/40823-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5d6f76da rejected from java.util.concurrent.ThreadPoolExecutor@3c7a2d55[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 14:55:53,345 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 14:55:53,345 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:53,345 ERROR [Listener at localhost/40823-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@d511565 rejected from java.util.concurrent.ThreadPoolExecutor@2bd02263[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 14:55:53,345 ERROR [Listener at localhost/40823-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@25b656ad rejected from java.util.concurrent.ThreadPoolExecutor@3c7a2d55[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 14:55:53,345 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41077,1675954443212 2023-02-09 14:55:53,345 ERROR [Listener at localhost/40823-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@759f80bd rejected from java.util.concurrent.ThreadPoolExecutor@2bd02263[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 14:55:53,346 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,41077,1675954443212] 2023-02-09 14:55:53,346 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41077,1675954443212; numProcessing=1 2023-02-09 14:55:53,347 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,41077,1675954443212 already deleted, retry=false 2023-02-09 14:55:53,347 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,41077,1675954443212 expired; onlineServers=2 2023-02-09 14:55:53,348 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,33553,1675954443180] 2023-02-09 14:55:53,348 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,33553,1675954443180; numProcessing=2 2023-02-09 14:55:53,349 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,33553,1675954443180 already deleted, retry=false 2023-02-09 14:55:53,349 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/namespace/0c1f8115bde7013387f9461b40485fe1/info/f8cd2ab33b4c4963b5452f2b9f3808ef, entries=2, sequenceid=6, filesize=4.8 K 2023-02-09 14:55:53,349 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,33553,1675954443180 expired; onlineServers=1 2023-02-09 14:55:53,351 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 0c1f8115bde7013387f9461b40485fe1 in 191ms, sequenceid=6, compaction requested=false 2023-02-09 14:55:53,354 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/namespace/0c1f8115bde7013387f9461b40485fe1/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-09 14:55:53,355 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:55:53,356 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675954445639.0c1f8115bde7013387f9461b40485fe1. 2023-02-09 14:55:53,356 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 0c1f8115bde7013387f9461b40485fe1: Waiting for close lock at 1675954553159Running coprocessor pre-close hooks at 1675954553159Disabling compacts and flushes for region at 1675954553159Disabling writes for close at 1675954553160 (+1 ms)Obtaining lock to block concurrent updates at 1675954553160Preparing flush snapshotting stores in 0c1f8115bde7013387f9461b40485fe1 at 1675954553160Finished memstore snapshotting hbase:namespace,,1675954445639.0c1f8115bde7013387f9461b40485fe1., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1675954553160Flushing stores of hbase:namespace,,1675954445639.0c1f8115bde7013387f9461b40485fe1. at 1675954553162 (+2 ms)Flushing 0c1f8115bde7013387f9461b40485fe1/info: creating writer at 1675954553162Flushing 0c1f8115bde7013387f9461b40485fe1/info: appending metadata at 1675954553168 (+6 ms)Flushing 0c1f8115bde7013387f9461b40485fe1/info: closing flushed file at 1675954553168Flushing 0c1f8115bde7013387f9461b40485fe1/info: reopening flushed file at 1675954553345 (+177 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for 0c1f8115bde7013387f9461b40485fe1 in 191ms, sequenceid=6, compaction requested=false at 1675954553351 (+6 ms)Writing region close event to WAL at 1675954553352 (+1 ms)Running coprocessor post-close hooks at 1675954553355 (+3 ms)Closed at 1675954553356 (+1 ms) 2023-02-09 14:55:53,356 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1675954445639.0c1f8115bde7013387f9461b40485fe1. 2023-02-09 14:55:53,357 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740/.tmp/rep_barrier/01fd5d661e6244d6b4185df4ddf76d70 2023-02-09 14:55:53,359 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 01fd5d661e6244d6b4185df4ddf76d70 2023-02-09 14:55:53,361 DEBUG [RS:0;jenkins-hbase4:33423] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-09 14:55:53,368 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740/.tmp/table/375567c4877e4e2abb003fbd880df238 2023-02-09 14:55:53,371 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 375567c4877e4e2abb003fbd880df238 2023-02-09 14:55:53,372 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740/.tmp/info/e9409d35ca234c7eb762af27e5bb4691 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740/info/e9409d35ca234c7eb762af27e5bb4691 2023-02-09 14:55:53,375 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e9409d35ca234c7eb762af27e5bb4691 2023-02-09 14:55:53,375 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740/info/e9409d35ca234c7eb762af27e5bb4691, entries=59, sequenceid=210, filesize=11.1 K 2023-02-09 14:55:53,376 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740/.tmp/rep_barrier/01fd5d661e6244d6b4185df4ddf76d70 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740/rep_barrier/01fd5d661e6244d6b4185df4ddf76d70 2023-02-09 14:55:53,379 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 01fd5d661e6244d6b4185df4ddf76d70 2023-02-09 14:55:53,379 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740/rep_barrier/01fd5d661e6244d6b4185df4ddf76d70, entries=24, sequenceid=210, filesize=7.5 K 2023-02-09 14:55:53,379 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740/.tmp/table/375567c4877e4e2abb003fbd880df238 as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740/table/375567c4877e4e2abb003fbd880df238 2023-02-09 14:55:53,382 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 375567c4877e4e2abb003fbd880df238 2023-02-09 14:55:53,382 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740/table/375567c4877e4e2abb003fbd880df238, entries=36, sequenceid=210, filesize=8.0 K 2023-02-09 14:55:53,383 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 223ms, sequenceid=210, compaction requested=false 2023-02-09 14:55:53,387 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-02-09 14:55:53,387 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-09 14:55:53,387 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-09 14:55:53,388 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-09 14:55:53,388 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675954553160Running coprocessor pre-close hooks at 1675954553160Disabling compacts and flushes for region at 1675954553160Disabling writes for close at 1675954553160Obtaining lock to block concurrent updates at 1675954553160Preparing flush snapshotting stores in 1588230740 at 1675954553160Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=81308, getHeapSize=128744, getOffHeapSize=0, getCellsCount=617 at 1675954553161 (+1 ms)Flushing stores of hbase:meta,,1.1588230740 at 1675954553162 (+1 ms)Flushing 1588230740/info: creating writer at 1675954553162Flushing 1588230740/info: appending metadata at 1675954553170 (+8 ms)Flushing 1588230740/info: closing flushed file at 1675954553170Flushing 1588230740/rep_barrier: creating writer at 1675954553344 (+174 ms)Flushing 1588230740/rep_barrier: appending metadata at 1675954553347 (+3 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1675954553347Flushing 1588230740/table: creating writer at 1675954553359 (+12 ms)Flushing 1588230740/table: appending metadata at 1675954553362 (+3 ms)Flushing 1588230740/table: closing flushed file at 1675954553362Flushing 1588230740/info: reopening flushed file at 1675954553373 (+11 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1675954553376 (+3 ms)Flushing 1588230740/table: reopening flushed file at 1675954553380 (+4 ms)Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 223ms, sequenceid=210, compaction requested=false at 1675954553383 (+3 ms)Writing region close event to WAL at 1675954553385 (+2 ms)Running coprocessor post-close hooks at 1675954553387 (+2 ms)Closed at 1675954553388 (+1 ms) 2023-02-09 14:55:53,388 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-09 14:55:53,445 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 14:55:53,445 INFO [RS:2;jenkins-hbase4:41077] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,41077,1675954443212; zookeeper connection closed. 2023-02-09 14:55:53,446 ERROR [Listener at localhost/40823-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@b0280b5 rejected from java.util.concurrent.ThreadPoolExecutor@2bd02263[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 14:55:53,446 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:41077-0x1003993358c0003, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 14:55:53,446 ERROR [Listener at localhost/40823-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6e9b2b59 rejected from java.util.concurrent.ThreadPoolExecutor@2bd02263[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 14:55:53,446 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3f8bd1a4] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3f8bd1a4 2023-02-09 14:55:53,546 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 14:55:53,546 INFO [RS:1;jenkins-hbase4:33553] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,33553,1675954443180; zookeeper connection closed. 2023-02-09 14:55:53,546 ERROR [Listener at localhost/40823-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3f2837a7 rejected from java.util.concurrent.ThreadPoolExecutor@3c7a2d55[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 14:55:53,546 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33553-0x1003993358c0002, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 14:55:53,546 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@44ece97c] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@44ece97c 2023-02-09 14:55:53,546 ERROR [Listener at localhost/40823-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3eb2fafb rejected from java.util.concurrent.ThreadPoolExecutor@3c7a2d55[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 14:55:53,561 INFO [RS:0;jenkins-hbase4:33423] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,33423,1675954443133; all regions closed. 2023-02-09 14:55:53,566 DEBUG [RS:0;jenkins-hbase4:33423] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/oldWALs 2023-02-09 14:55:53,566 INFO [RS:0;jenkins-hbase4:33423] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33423%2C1675954443133.meta:.meta(num 1675954445370) 2023-02-09 14:55:53,570 DEBUG [RS:0;jenkins-hbase4:33423] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/oldWALs 2023-02-09 14:55:53,570 INFO [RS:0;jenkins-hbase4:33423] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C33423%2C1675954443133:(num 1675954444843) 2023-02-09 14:55:53,570 DEBUG [RS:0;jenkins-hbase4:33423] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:55:53,571 INFO [RS:0;jenkins-hbase4:33423] regionserver.LeaseManager(133): Closed leases 2023-02-09 14:55:53,571 INFO [RS:0;jenkins-hbase4:33423] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-02-09 14:55:53,571 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 14:55:53,572 INFO [RS:0;jenkins-hbase4:33423] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33423 2023-02-09 14:55:53,574 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-09 14:55:53,574 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33423,1675954443133 2023-02-09 14:55:53,574 ERROR [Listener at localhost/40823-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4ecbcc3d rejected from java.util.concurrent.ThreadPoolExecutor@4af4513c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 14:55:53,575 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,33423,1675954443133] 2023-02-09 14:55:53,575 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,33423,1675954443133; numProcessing=3 2023-02-09 14:55:53,576 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,33423,1675954443133 already deleted, retry=false 2023-02-09 14:55:53,576 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,33423,1675954443133 expired; onlineServers=0 2023-02-09 14:55:53,576 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,43829,1675954441993' ***** 2023-02-09 14:55:53,576 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-09 14:55:53,577 DEBUG [M:0;jenkins-hbase4:43829] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@32e59004, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-09 14:55:53,577 INFO [M:0;jenkins-hbase4:43829] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,43829,1675954441993 2023-02-09 14:55:53,577 INFO [M:0;jenkins-hbase4:43829] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,43829,1675954441993; all regions closed. 2023-02-09 14:55:53,577 DEBUG [M:0;jenkins-hbase4:43829] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-09 14:55:53,577 DEBUG [M:0;jenkins-hbase4:43829] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-09 14:55:53,577 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-09 14:55:53,577 DEBUG [M:0;jenkins-hbase4:43829] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-09 14:55:53,577 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675954444344] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675954444344,5,FailOnTimeoutGroup] 2023-02-09 14:55:53,577 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675954444344] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675954444344,5,FailOnTimeoutGroup] 2023-02-09 14:55:53,577 INFO [M:0;jenkins-hbase4:43829] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-09 14:55:53,577 INFO [M:0;jenkins-hbase4:43829] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-09 14:55:53,577 INFO [M:0;jenkins-hbase4:43829] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-09 14:55:53,577 DEBUG [M:0;jenkins-hbase4:43829] master.HMaster(1502): Stopping service threads 2023-02-09 14:55:53,578 INFO [M:0;jenkins-hbase4:43829] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-09 14:55:53,578 ERROR [M:0;jenkins-hbase4:43829] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-02-09 14:55:53,578 INFO [M:0;jenkins-hbase4:43829] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-09 14:55:53,578 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-09 14:55:53,578 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-09 14:55:53,578 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-09 14:55:53,578 DEBUG [M:0;jenkins-hbase4:43829] zookeeper.ZKUtil(398): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-09 14:55:53,578 WARN [M:0;jenkins-hbase4:43829] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-09 14:55:53,578 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-09 14:55:53,578 INFO [M:0;jenkins-hbase4:43829] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-09 14:55:53,579 INFO [M:0;jenkins-hbase4:43829] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-09 14:55:53,579 DEBUG [M:0;jenkins-hbase4:43829] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-09 14:55:53,579 INFO [M:0;jenkins-hbase4:43829] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 14:55:53,579 DEBUG [M:0;jenkins-hbase4:43829] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 14:55:53,579 DEBUG [M:0;jenkins-hbase4:43829] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-09 14:55:53,579 DEBUG [M:0;jenkins-hbase4:43829] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 14:55:53,579 INFO [M:0;jenkins-hbase4:43829] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=607.95 KB heapSize=719.01 KB 2023-02-09 14:55:53,588 INFO [M:0;jenkins-hbase4:43829] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=607.95 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/05786fba3fdb4d58adbfd0e611755faf 2023-02-09 14:55:53,591 INFO [M:0;jenkins-hbase4:43829] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 05786fba3fdb4d58adbfd0e611755faf 2023-02-09 14:55:53,591 DEBUG [M:0;jenkins-hbase4:43829] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/05786fba3fdb4d58adbfd0e611755faf as hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/05786fba3fdb4d58adbfd0e611755faf 2023-02-09 14:55:53,594 INFO [M:0;jenkins-hbase4:43829] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 05786fba3fdb4d58adbfd0e611755faf 2023-02-09 14:55:53,594 INFO [M:0;jenkins-hbase4:43829] regionserver.HStore(1080): Added hdfs://localhost:38049/user/jenkins/test-data/a12c82a6-4c4a-b14d-8f5c-b467a14e36d9/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/05786fba3fdb4d58adbfd0e611755faf, entries=156, sequenceid=1282, filesize=34.9 K 2023-02-09 14:55:53,595 INFO [M:0;jenkins-hbase4:43829] regionserver.HRegion(2947): Finished flush of dataSize ~607.95 KB/622541, heapSize ~718.99 KB/736248, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 16ms, sequenceid=1282, compaction requested=false 2023-02-09 14:55:53,595 INFO [M:0;jenkins-hbase4:43829] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-09 14:55:53,595 DEBUG [M:0;jenkins-hbase4:43829] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675954553579Disabling compacts and flushes for region at 1675954553579Disabling writes for close at 1675954553579Obtaining lock to block concurrent updates at 1675954553579Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1675954553579Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=622541, getHeapSize=736248, getOffHeapSize=0, getCellsCount=1499 at 1675954553579Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1675954553579Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1675954553580 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1675954553583 (+3 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1675954553583Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1675954553592 (+9 ms)Finished flush of dataSize ~607.95 KB/622541, heapSize ~718.99 KB/736248, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 16ms, sequenceid=1282, compaction requested=false at 1675954553595 (+3 ms)Writing region close event to WAL at 1675954553595Closed at 1675954553595 2023-02-09 14:55:53,599 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-09 14:55:53,599 INFO [M:0;jenkins-hbase4:43829] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-09 14:55:53,600 INFO [M:0;jenkins-hbase4:43829] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:43829 2023-02-09 14:55:53,602 DEBUG [M:0;jenkins-hbase4:43829] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,43829,1675954441993 already deleted, retry=false 2023-02-09 14:55:53,675 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 14:55:53,675 INFO [RS:0;jenkins-hbase4:33423] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,33423,1675954443133; zookeeper connection closed. 2023-02-09 14:55:53,675 ERROR [Listener at localhost/40823-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2e0953ee rejected from java.util.concurrent.ThreadPoolExecutor@4af4513c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 14:55:53,676 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@19968595] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@19968595 2023-02-09 14:55:53,676 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): regionserver:33423-0x1003993358c0001, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 14:55:53,676 ERROR [Listener at localhost/40823-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5626037d rejected from java.util.concurrent.ThreadPoolExecutor@4af4513c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 14:55:53,676 INFO [Listener at localhost/40823] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-09 14:55:53,775 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 14:55:53,775 INFO [M:0;jenkins-hbase4:43829] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,43829,1675954441993; zookeeper connection closed. 2023-02-09 14:55:53,775 ERROR [Listener at localhost/40823-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2addf990 rejected from java.util.concurrent.ThreadPoolExecutor@5cd3fd[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 14:55:53,776 DEBUG [Listener at localhost/40823-EventThread] zookeeper.ZKWatcher(600): master:43829-0x1003993358c0000, quorum=127.0.0.1:52509, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-09 14:55:53,776 ERROR [Listener at localhost/40823-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@38a985ca rejected from java.util.concurrent.ThreadPoolExecutor@5cd3fd[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-09 14:55:53,777 WARN [Listener at localhost/40823] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-09 14:55:53,781 INFO [Listener at localhost/40823] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 14:55:53,866 ERROR [BP-2069929948-172.31.14.131-1675954438937 heartbeating to localhost/127.0.0.1:38049] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-2069929948-172.31.14.131-1675954438937 (Datanode Uuid 89217ea8-d9db-4980-9a81-2c27c9377629) service to localhost/127.0.0.1:38049 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-09 14:55:53,883 WARN [BP-2069929948-172.31.14.131-1675954438937 heartbeating to localhost/127.0.0.1:38049] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-2069929948-172.31.14.131-1675954438937 (Datanode Uuid 30e687f1-c071-43d7-89db-78c265b7458a) service to localhost/127.0.0.1:38049 2023-02-09 14:55:53,885 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/cluster_fa49ce5e-dcbf-0fd7-6b70-a45a30f7cf80/dfs/data/data5/current/BP-2069929948-172.31.14.131-1675954438937] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 14:55:53,886 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/cluster_fa49ce5e-dcbf-0fd7-6b70-a45a30f7cf80/dfs/data/data6/current/BP-2069929948-172.31.14.131-1675954438937] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 14:55:53,886 WARN [Listener at localhost/40823] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-09 14:55:53,889 INFO [Listener at localhost/40823] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 14:55:53,995 WARN [BP-2069929948-172.31.14.131-1675954438937 heartbeating to localhost/127.0.0.1:38049] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-09 14:55:53,996 WARN [BP-2069929948-172.31.14.131-1675954438937 heartbeating to localhost/127.0.0.1:38049] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-2069929948-172.31.14.131-1675954438937 (Datanode Uuid f8e8dad4-c7b5-422d-a273-bab2cf112e44) service to localhost/127.0.0.1:38049 2023-02-09 14:55:53,997 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/cluster_fa49ce5e-dcbf-0fd7-6b70-a45a30f7cf80/dfs/data/data3/current/BP-2069929948-172.31.14.131-1675954438937] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 14:55:53,997 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/cluster_fa49ce5e-dcbf-0fd7-6b70-a45a30f7cf80/dfs/data/data4/current/BP-2069929948-172.31.14.131-1675954438937] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 14:55:53,997 WARN [Listener at localhost/40823] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-09 14:55:54,004 INFO [Listener at localhost/40823] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 14:55:54,108 WARN [BP-2069929948-172.31.14.131-1675954438937 heartbeating to localhost/127.0.0.1:38049] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-2069929948-172.31.14.131-1675954438937 (Datanode Uuid 89217ea8-d9db-4980-9a81-2c27c9377629) service to localhost/127.0.0.1:38049 2023-02-09 14:55:54,109 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/cluster_fa49ce5e-dcbf-0fd7-6b70-a45a30f7cf80/dfs/data/data1/current/BP-2069929948-172.31.14.131-1675954438937] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 14:55:54,109 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4bede8e1-cf41-77ac-c2c4-4c426b89b3fe/cluster_fa49ce5e-dcbf-0fd7-6b70-a45a30f7cf80/dfs/data/data2/current/BP-2069929948-172.31.14.131-1675954438937] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-09 14:55:54,137 INFO [Listener at localhost/40823] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-09 14:55:54,195 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-09 14:55:54,327 INFO [Listener at localhost/40823] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-09 14:55:54,551 INFO [Listener at localhost/40823] hbase.HBaseTestingUtility(1293): Minicluster is down